-- Logs begin at Wed 2022-08-24 17:10:34 UTC, end at Wed 2022-08-24 18:06:48 UTC. -- Aug 24 17:34:04.617404 dv-cinder-840525-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 17:34:04.627604 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 17:34:04 2022] *** Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: nodename: dv-cinder-840525-3-iscsi Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: machine: x86_64 Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: clock source: unix Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: pcre jit disabled Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: detected number of CPU cores: 4 Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: current working directory: / Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: detected binary path: /usr/bin/uwsgi-core Aug 24 17:34:04.629646 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: your processes number limit is 47680 Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: your memory page size is 4096 bytes Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: detected max file descriptor number: 2048 Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: lock engine: pthread robust mutexes Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: thunder lock: enabled Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 17:34:04.630248 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 17:34:04.669240 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: Python main interpreter initialized at 0x55bdf9c629c0 Aug 24 17:34:04.669240 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: python threads support enabled Aug 24 17:34:04.669240 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: your server socket listen backlog is limited to 100 connections Aug 24 17:34:04.669240 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: your mercy for graceful operations on workers is 90 seconds Aug 24 17:34:04.669579 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 17:34:04.669649 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: *** Operational MODE: preforking *** Aug 24 17:34:04.669924 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: *** uWSGI is running in multiple interpreter mode *** Aug 24 17:34:04.669924 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: spawned uWSGI master process (pid: 87974) Aug 24 17:34:04.670120 dv-cinder-840525-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 17:34:04.670847 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: spawned uWSGI worker 1 (pid: 87987, cores: 1) Aug 24 17:34:04.672857 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: spawned uWSGI worker 2 (pid: 87988, cores: 1) Aug 24 17:34:04.672857 dv-cinder-840525-3-iscsi devstack@keystone.service[87974]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 17:34:08.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:34:08.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: from cryptography.utils import int_from_bytes Aug 24 17:34:08.988525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:34:08.988525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: from cryptography.utils import int_from_bytes Aug 24 17:34:08.988966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:34:08.988966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: from cryptography.utils import int_from_bytes Aug 24 17:34:08.989097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:34:08.989097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: from cryptography.utils import int_from_bytes Aug 24 17:34:09.039105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:34:09.039282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:34:09.039282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:34:09.039433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:34:09.039537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] command line args: [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:34:09.039617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] command line args: [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:34:09.039747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:34:09.039839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:34:09.039912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ================================================================================ {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:34:09.039987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ================================================================================ {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:34:09.040164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] admin_token = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] admin_token = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] config_dir = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] config_dir = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] config_source = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] config_source = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.040944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] debug = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] debug = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.041922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_config_append = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_config_append = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.042938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_dir = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_dir = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_file = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_file = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.043971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.044047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.044172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.044879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.044960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.045917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.046915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] publish_errors = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] publish_errors = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.047976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.048952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] standard_threads = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] standard_threads = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] use_journal = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] use_journal = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.049939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] use_json = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] use_json = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] use_stderr = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] use_stderr = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] use_syslog = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] use_syslog = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:34:09.050740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.050817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.050978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.051986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.052961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.053929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.054993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.055930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.056929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.057947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.058915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.059859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.060578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.060656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.060726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.060840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.060928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.061990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.062937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.063967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.064896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.065978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.066933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.067936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.068984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.069869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.070892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.071963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.072876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.073958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.074996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.075914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.076957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.external = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.external = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.077880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.password = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.password = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] auth.token = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] auth.token = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.078924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.079983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.080898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.081993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.082134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.088920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.089410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.090902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.091865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.092757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.093987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.094850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.095843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.096947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.097930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.098938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.099973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.100969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.101998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.102998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.103945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.104992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.105960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.106720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.107419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.107522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.107686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.107768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.107921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.108953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.109945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.110912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.111889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] role.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] role.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.112863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.113976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.114943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.115941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.116944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] role.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] role.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.117923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.118884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.119894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.120843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.121980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.122934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.123923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.124956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.connection = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.125935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.126885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.127880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.128976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:34:09.129710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.129914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.130156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.130455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.130719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.connection = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.130965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.131213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.131453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.131729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.131965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.132205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.132441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.132679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.132925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.133164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.133408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.133645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.133893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.134130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.134377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.134608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.134850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.135365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:34:09.135564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:34:09.159494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.159727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.159952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.160393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.160628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.160853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.168570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.169288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.197644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.197878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.197964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.198188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.253554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.253687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.303580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.303698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.303832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.303931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.463934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.464018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.464154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.464247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.464370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.464453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.489937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.490929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.501675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.502005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.515979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.516944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:34:09.533980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.534190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:34:09.534535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.534653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.534736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:34:09.534868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:34:09.534997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.535214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.535304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.535418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:34:09.535724 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.535807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.535992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.536085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.536516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.536786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.546218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.546497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.546742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.546838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.547026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.547109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.547299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.547615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.554012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.554592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.556926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.557213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.557500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.557591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.557787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.557870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.558962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.559153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.559261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.559435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.559561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.559907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.560501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.560667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.560945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.582760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.583420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.584243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.584527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.584924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.585210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.589192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.589478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.589762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.589885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.590174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.590466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.596144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.596431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.596913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.597203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.600525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.600817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.601101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.601327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.601421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.601625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.601718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.601906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.602998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.603277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.620498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.620782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.621225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.621513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.625201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.625965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.628441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.628743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.629225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.629527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.632954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.633753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.636122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.636934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.639373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.639701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.639992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.640209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.640516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.640608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.640811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.640903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.641103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.641205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.641396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.641704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.656831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.657129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.657399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.657491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.657663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.657786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.658061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.658329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.664574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.665339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.666096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.666386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.667113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.667446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.669065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.669876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.670179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.670472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.670751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.671041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.671332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.671655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.671953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.672238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.682101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.682385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.682673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.682953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.683235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.683538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.683626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.683820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.683907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.684127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.684415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.684709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.685536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.685821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.699704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.701341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.702813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.704422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.705729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.707166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.707303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.708670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.708801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.708953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.709256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.710226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.710512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.710818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.715731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.717218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.717396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.718865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.720278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.720571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.720858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.721135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.722069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.722367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.722655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.722935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.729075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.729367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.729661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.730988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.731285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.731602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.737413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.737697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.737978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.738255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.738538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.738821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:34:09.739363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.739968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.740985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.741074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.741255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.741539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.741819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.742106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.742408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.742690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.742955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:34:09.776579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bdf9c629c0 pid: 87988 (default app) Aug 24 17:34:09.778298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bdf9c629c0 pid: 87987 (default app) Aug 24 17:34:09.780576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3a0f-5d37-47d8-a527-68be3733126a None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:09.780873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3a0f-5d37-47d8-a527-68be3733126a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:09.781148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f3a0f-5d37-47d8-a527-68be3733126a None None] PATH_INFO: `/v3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:09.785309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Wed Aug 24 17:34:09 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:16.337792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:16.338322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:16.338322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:16.343205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:16.349554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:16.349955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:16.350101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:16.421919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: INFO dbcounter [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] Registered counter for database keystone Aug 24 17:34:16.440690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG oslo_db.sqlalchemy.engines [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87988) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:34:16.446802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writer thread running {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:34:16.626202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG passlib.handlers.bcrypt [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87988) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:34:16.626543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG passlib.handlers.bcrypt [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87988) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:34:16.681229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:16.682835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:16.810708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:34:17.246726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:17.315071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: INFO dbcounter [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] Registered counter for database keystone Aug 24 17:34:17.336834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG oslo_db.sqlalchemy.engines [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87987) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:34:17.343612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writer thread running {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:34:17.461789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191d34-e356-4646-a8f5-467ffd17a1ed None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:17.466084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:17.466948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:17.467259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:17.467259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:17.475244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:17.527307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:34:17.527683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91220b-77cf-48c1-beb1-6303a81ac106 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:17.538360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 24 17:34:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:34:17.544409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:17.559344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e2bc8-bc28-41d4-9c36-187cfeea2845 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:17.563880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:17.564992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:17.565277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:17.565550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] PATH_INFO: `/v3/projects/c0a52b9c017649d7b6608817068a47f8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:17.567113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] RBAC: Authorizing `identity:get_project(project_id=c0a52b9c017649d7b6608817068a47f8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:17.620401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:34:17.620748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1337ba-4011-4be3-9e31-591beed1b410 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:17.625300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Wed Aug 24 17:34:17 2022] GET /identity/v3/projects/c0a52b9c017649d7b6608817068a47f8 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:19.501628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebbe72-4678-4cac-b2be-e45d554bdf66 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:19.502201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebbe72-4678-4cac-b2be-e45d554bdf66 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:19.502201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebbe72-4678-4cac-b2be-e45d554bdf66 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:19.506291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:19.511802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:19.512201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:19.512365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:19.533319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:19.534861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:19.542744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:34:19.806470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebbe72-4678-4cac-b2be-e45d554bdf66 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:19.818925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebbe72-4678-4cac-b2be-e45d554bdf66 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:19.823093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:19.823707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:19.823986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:19.824257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:19.832348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:19.834133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca6297-0ec4-4d2f-a779-32c27fa2dc19 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:19.842656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Aug 24 17:34:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:34:19.847859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:19.856832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415c138-17b9-4bb1-892f-878fdc43945f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:19.861595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:19.862215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:19.862492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:19.862763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:19.864111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:19.865929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13934999-fe83-4354-b94b-248701bd06b9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:19.867438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Wed Aug 24 17:34:19 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:21.838601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1723cf-56b3-404b-89c0-6a96fff838d3 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.839111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1723cf-56b3-404b-89c0-6a96fff838d3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.839111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1723cf-56b3-404b-89c0-6a96fff838d3 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.840596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:21.843753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac1c7-d2db-4bb1-b2ca-02273925a721 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.844127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac1c7-d2db-4bb1-b2ca-02273925a721 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.844420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac1c7-d2db-4bb1-b2ca-02273925a721 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.845803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:21.845929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.846204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.846486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.850951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:21.851239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:21.851543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:21.869848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:21.871024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:21.889509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG passlib.handlers.bcrypt [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87987) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:34:21.889925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG passlib.handlers.bcrypt [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87987) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:34:21.936474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:21.938076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:21.951144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 7/12] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:22.035016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 6/13] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:22.223186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.247289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.311657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.327003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fc3ff-2428-4653-af5b-c92b3c50667c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.359955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.360848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.361126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.361399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.373250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:22.381710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:22.382666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:22.383751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 8/14] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:22.388732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.414341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c09113-a42d-4206-a9cf-6d0ff0284d21 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.416646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.453129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.453994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.454368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.454551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.471845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.490847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40c9bc-3531-49d7-a8e0-1111929fd0c2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.522329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.523904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.525986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.526603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.526887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.527160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.527857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:22.528160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.530118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.542079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 9/15] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:22.547761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.624455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 7/16] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:34:22.629978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.632673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.647062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f925fe3-a138-4267-83de-2f04b14ee7b3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.680241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.680878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.681158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.681436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.682574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.684067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.686173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 10/17] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:22.691884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.729472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24bf06-c649-4477-84cd-6d3164bdd59c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.763075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.764552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.764832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.765102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.783810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.785279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.791812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e3fea-fd6f-41aa-a58f-62fa9adf6e4e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:34:22.804003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:22.805060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:34:22.805942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:34:22.805942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:34:22.805942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:22.805942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 8/18] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:34:22.810950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.826590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.827228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.827536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.827815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.828942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.830392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.831975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 11/19] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:22.837762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.867224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.910961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4419-9568-42a5-ae34-be99ee58f752 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.918298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.938316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfa489-32e4-4384-8f86-c75a6021b23c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:22.948361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.949281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.949723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.949996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.950761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:22.951068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.952513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e57a-2aef-4ce7-96d1-6ea9512886e6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.969740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 9/20] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:22.971983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:22.972597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:22.972876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:22.973145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:22.973874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.domain.id=default, user.id=6c7d397143574599b62017bc352e1183) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:22.974179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:22.975673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8e6f6-1a5a-490f-9ae1-1057fc5bb13e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:22.984313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 12/21] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/role_assignments?user.id=6c7d397143574599b62017bc352e1183&scope.domain.id=default&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:25.155812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ccfb-4371-4e31-9bd2-f3fa08744c1d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.157261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ccfb-4371-4e31-9bd2-f3fa08744c1d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.157764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ccfb-4371-4e31-9bd2-f3fa08744c1d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.160162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 10/22] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:25.166051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f086-2648-4fc6-8630-dbeadafc9cae None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.166875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f086-2648-4fc6-8630-dbeadafc9cae None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.167357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f086-2648-4fc6-8630-dbeadafc9cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.196809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a241f086-2648-4fc6-8630-dbeadafc9cae None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:25.198326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a241f086-2648-4fc6-8630-dbeadafc9cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.293679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 13/23] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:25.467710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238fdd-7126-430e-9db7-0974ab772862 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.468341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238fdd-7126-430e-9db7-0974ab772862 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.468836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2238fdd-7126-430e-9db7-0974ab772862 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.470653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 11/24] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:25.475779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.476295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.476769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.500575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:25.501809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.526928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285176-49f8-41d8-b7e0-fb4df00459ef None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.533454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285176-49f8-41d8-b7e0-fb4df00459ef None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.534042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285176-49f8-41d8-b7e0-fb4df00459ef None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.536105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 12/25] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:25.541653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840ded-8f9a-4c9e-b9d3-9d0237f24fd6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.542186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840ded-8f9a-4c9e-b9d3-9d0237f24fd6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.542645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840ded-8f9a-4c9e-b9d3-9d0237f24fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.584591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-50840ded-8f9a-4c9e-b9d3-9d0237f24fd6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:25.586171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-50840ded-8f9a-4c9e-b9d3-9d0237f24fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.610089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 14/26] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:25.682148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 13/27] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:25.789063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:25.792802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:25.847343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df57ff-ce94-4770-bb9d-b67707ef039e None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.847993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df57ff-ce94-4770-bb9d-b67707ef039e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.848476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df57ff-ce94-4770-bb9d-b67707ef039e None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.850005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 14/28] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:25.869029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.869605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.870093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.915552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7ae15-5868-4c01-8bd3-0c21a01b15b4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:25.925363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:25.926802 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:25.960188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:25.960837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:25.961115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:25.961391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:25.973753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:25.975254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:34:26.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:26.009566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:34:26.010358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:34:26.010358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:34:26.010358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:26.010358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 15/29] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:26.016465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.036681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.067602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 15/30] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 17:34:26.067602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.123841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.189857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7921d4-a45d-4108-aee1-a3347b9e59a3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.223857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.224464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.224798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.225072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.225711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:26.226006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:26.227386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.227670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018117b6-dccd-43d8-8707-55ae26d4d187 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.245546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:26.246401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.291558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.292228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.292583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.292860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.293519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:26.295450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.433718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9f4e6-93d1-4f9a-a26f-f907ca4ffb44 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.452478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.473083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.473726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.474065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.474394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.484860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 16/32] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 17:34:26 2022] POST /identity/v3/roles => generated 219 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:26.486370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.486741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:26.493484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:26.495776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:26.497719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.577176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.686244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a5e3-39aa-48e7-a7e8-7b7de34f5acf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.690162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416e980-93ad-4742-a915-10bb99edac0f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:26.729139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.729767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.730042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.730311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.731213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:26.731577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:26.732266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.733057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.733507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.734327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:26.734463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.736258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499bb20-e09e-4aa1-afe9-a775d178d243 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:26.781014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 17/34] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:34:26 2022] POST /identity/v3/projects => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:26.782112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:26.883736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 18/35] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 17:34:26 2022] POST /identity/v3/roles => generated 225 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:26.885274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57322-f5f2-40de-a283-508adc0287d3 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.886823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57322-f5f2-40de-a283-508adc0287d3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.886823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57322-f5f2-40de-a283-508adc0287d3 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.896018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 19/36] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:26.897793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d84d-6fdd-4533-a478-3616d2abd1dc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:26.898259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d84d-6fdd-4533-a478-3616d2abd1dc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:26.898711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245d84d-6fdd-4533-a478-3616d2abd1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:26.921822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4245d84d-6fdd-4533-a478-3616d2abd1dc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:26.923762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4245d84d-6fdd-4533-a478-3616d2abd1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:26.961940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.968498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:26.986538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06d039-0941-40b4-904a-fb624fe5eb03 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.017684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 20/37] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:27.018793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d016a0-0452-4ee0-9209-bb8ef80eb2f3 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.019083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d016a0-0452-4ee0-9209-bb8ef80eb2f3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.019364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d016a0-0452-4ee0-9209-bb8ef80eb2f3 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.030285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 21/38] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:27.033714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.034692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.034940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.035215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.035972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:27.036329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.037702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.040944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.041237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.041520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.056488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 18/39] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:27.064734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.068527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:27.070501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:27.151179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 22/40] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:27.190484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5ad60-9f97-4b34-8185-a55fbf2db383 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.225467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.226103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.226379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.226649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.228037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.229490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.231061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 19/41] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:27 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:27.236919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.304156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.324737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:27.337406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:27.345057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501ff24-9e6c-4dec-a1bc-347884c374f4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.416217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b581e-bd8b-43bd-88d9-5f5a7868585d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.453652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.454411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.454707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.454993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.456184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.457661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.459282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 20/42] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:27.460072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.492756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.493316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.493603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.493881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.494993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.496620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.498823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 23/43] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:27.499602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.549216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:27.564523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d437105-10a3-4419-ab21-5a5e29ede3eb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.597816 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.598423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.598698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.598981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.599704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.601076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.607096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63d977-f12e-4e4d-b4a3-f546e156de10 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.615437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 21/44] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 17:34:27 2022] POST /identity/v3/roles => generated 223 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:27.616240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:27.645157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.645828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.646133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.646413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.647115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.648640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8269d13-79b4-44fb-b086-641dc942eef9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.681197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 24/45] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:34:27 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:27.739922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8920f-a6fe-4943-aa65-ac585a977d63 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:27.784104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:27.784679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:27.784965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:27.785245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] PATH_INFO: `/v3/domains/default/users/6c7d397143574599b62017bc352e1183/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:27.787291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6c7d397143574599b62017bc352e1183, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:27.788815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17e839-0744-43c9-84eb-c04d0c0dfd38 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:27.806975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 22/46] 10.222.0.27 () {58 vars in 1361 bytes} [Wed Aug 24 17:34:27 2022] PUT /identity/v3/domains/default/users/6c7d397143574599b62017bc352e1183/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:29.656437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64cc6-caf3-4521-93f0-c8fe92f91a69 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:29.656945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64cc6-caf3-4521-93f0-c8fe92f91a69 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:29.657028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64cc6-caf3-4521-93f0-c8fe92f91a69 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:29.658323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 25/47] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:29.663886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:29.664277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:29.664545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:29.677233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:29.683104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:29.684802 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:29.693911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140267-fd6d-410d-a9cf-e06f56bb713e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:29.694200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140267-fd6d-410d-a9cf-e06f56bb713e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:29.694846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1140267-fd6d-410d-a9cf-e06f56bb713e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:29.696559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 26/48] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:29.702404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:29.702689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:29.703257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:29.721765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:29.722972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:29.734637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:29.754884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 23/49] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:29.791868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 27/50] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:30.028879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.058665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.127106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796ec23-a547-4513-ae19-3635b3b92dcb None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.156973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf5cb-c8f2-4a61-93d1-64101aa2a570 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.160957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.161595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.161877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.162153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.163288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.164800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.166551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 24/51] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:30.171905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.175313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:30.191031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.191724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.191990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.192264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.202950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:30.207939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:30.208779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:30.209537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 28/52] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:34:30.216134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.225650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:30.274578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df50699-f792-41cd-b5e1-40a4653982d8 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.307656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.308259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.308533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.308856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.309430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.310903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.315984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e409b0-a068-4733-8c91-01e43c6662b5 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.336347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 25/53] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 24 17:34:30 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:34:30.351223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.351966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.352248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.352518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.353197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:30.353515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.354943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.365554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 29/54] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:30.371651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.431177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:30.477600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430471f-4f88-481d-9b86-263a34e39b3c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.511953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.512589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.512869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.513145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.514319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.515834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.517443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 26/55] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:30.524177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.592514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:30.627313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49d3c1-7c79-47b9-8e05-4efc25b44404 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.660688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.661330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.661605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.661884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.663019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.664499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5efa0-c58f-474e-8735-73438259a38d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.666243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 30/56] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:30.672354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:30.771024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7353d4a-9989-42e4-99ae-0d42dbde49a3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:30.793693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:30.804754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:30.805367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:30.805640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:30.805905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:30.806641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.domain.id=default, user.id=6c7d397143574599b62017bc352e1183) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:30.806925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:30.808393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c833e1-e2c3-4828-9e3f-c4ab6a07b903 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:30.816770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 27/57] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Aug 24 17:34:30 2022] GET /identity/v3/role_assignments?user.id=6c7d397143574599b62017bc352e1183&scope.domain.id=default&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:32.262007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f06a-c652-46dd-b9b4-66ea71f37ac6 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:32.262524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f06a-c652-46dd-b9b4-66ea71f37ac6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:32.262633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610f06a-c652-46dd-b9b4-66ea71f37ac6 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:32.264460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 31/58] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:32.269563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:32.269970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:32.270118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:32.288543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:32.289748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:32.354001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 28/59] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:32.621272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f06a-c652-46dd-b9b4-66ea71f37ac6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:32.723559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610f06a-c652-46dd-b9b4-66ea71f37ac6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:32.755601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:32.756217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:32.756493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:32.756760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:32.757847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:32.759271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17097853-e0ad-4e75-a2bf-1f59c61bdcd8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:32.760832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 32/60] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:32.766790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:32.862228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cf758-4bce-4f18-bc58-bf957633537c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:32.878506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:32.895227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:32.895855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:32.896134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:32.896406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:32.897052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:32.898493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d0b2e-969d-410d-8beb-c62441f0f1fb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:32.924886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 29/61] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:32 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:34:35.882139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d6b2-9e3e-44ae-bfcc-70abb1e69f8f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:35.882139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d6b2-9e3e-44ae-bfcc-70abb1e69f8f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:35.882139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576d6b2-9e3e-44ae-bfcc-70abb1e69f8f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:35.882139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 33/62] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:35.887210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265da8-4945-4db8-826e-d234c6cf49bb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:35.888062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265da8-4945-4db8-826e-d234c6cf49bb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:35.888570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265da8-4945-4db8-826e-d234c6cf49bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:35.911683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ba265da8-4945-4db8-826e-d234c6cf49bb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:35.913222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ba265da8-4945-4db8-826e-d234c6cf49bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:35.998604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 30/63] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:36.003863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad31b8-2b41-4d87-af69-92ebcb5531e6 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.004399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad31b8-2b41-4d87-af69-92ebcb5531e6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.004888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad31b8-2b41-4d87-af69-92ebcb5531e6 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.006556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 34/64] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:36.011703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.012209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.012698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.037743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:36.039250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:36.097541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57781a9-99c8-47f0-8982-5aca0ac2aca8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.098162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57781a9-99c8-47f0-8982-5aca0ac2aca8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.098661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57781a9-99c8-47f0-8982-5aca0ac2aca8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.100247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 35/65] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:36.110087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.110903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.111380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.118894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.132404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:36.133963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 31/66] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:36.134405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:36.226744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 36/67] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:36.287607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.331966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.421947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995e4b-2725-473a-8560-1d422c2f2170 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:36.462371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:36.463506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.463811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.464090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.476190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:36.490226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:36.589884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:36.590900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:36.592280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 32/68] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:36.592923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.628786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8048a4d-4ed8-4c22-bfdc-3012085bd06a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:36.665147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:36.666124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.666631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.667120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.673121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.680995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:36.686541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:36.687392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:36.688186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 37/69] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:36.688862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.693095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.698285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bbb91-54f1-460e-a569-71db742499e9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:36.734774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:36.735417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.735994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.736268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.747083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-54e8856b-1971-44c5-a1f5-7ab85cf53ac6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:36.752040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:36.752892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:36.753688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 33/70] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:36.754825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f74a-58ac-486f-aedf-cb3738897718 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.754825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f74a-58ac-486f-aedf-cb3738897718 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.754825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027f74a-58ac-486f-aedf-cb3738897718 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.756272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 34/71] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:36.756767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f74a-58ac-486f-aedf-cb3738897718 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.808653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2360e7e-bf53-4b93-8d19-458a080f0270 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:36.846890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:36.847522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.848120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.848387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.849042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:36.849500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:36.850876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:36.859332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 38/72] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:36.860310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:36.874502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027f74a-58ac-486f-aedf-cb3738897718 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:36.908973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:36.909607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.909860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.910133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.910792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:36.911100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:36.912539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c50cd-05eb-4184-b2cb-075caad6934c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:36.920109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.920903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 35/73] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:36.922174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:36.922455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:36.922751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:36.941146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:36.942302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:36.953604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:36.968653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ddd43-93c2-4ead-9b24-4a5bfc2bda54 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.002199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.002806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.003087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.003361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.004042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:37.004343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.005736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d9dfc-9d87-4f66-b687-c7201f60c5b7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.009775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 36/74] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:37.010608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.017394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 39/75] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:34:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:37.018581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317b4b-22f0-48a5-98fc-6f003f9da117 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.018870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317b4b-22f0-48a5-98fc-6f003f9da117 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.019147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d317b4b-22f0-48a5-98fc-6f003f9da117 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.020552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 40/76] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:37.022542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d317b4b-22f0-48a5-98fc-6f003f9da117 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.129551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d317b4b-22f0-48a5-98fc-6f003f9da117 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.130397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca69ebd-6e73-4273-ab2a-7f669344dce9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.147757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.165850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.166689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.166974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.167251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.167967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.168435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.168608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.168883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.169152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.170042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.177030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 41/77] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:37.177030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.202374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.204115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50896f52-0d78-4da0-ae1b-6d0f9a9f0325 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.205979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 37/78] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:37.207115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.213675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.213965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.223515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.235266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:37.236788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:37.287634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dda884-b796-4f11-bc42-3aa09551c4b8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.304357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 38/79] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:37.305203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.325634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.327559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.328335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.328763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.331555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.334090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.336157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 42/80] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:37.337493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.401846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.407887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbfc53-a4a2-4d5f-be34-79a968ef6ec7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.445046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.445941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.446227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.446504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.450210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.450469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707a4f-27ad-4211-8580-9e4d195fd8a2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.486776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.487390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.487719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.487995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.523728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.525265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.573872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.575342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.629968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.630338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 39/81] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:37.631194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.664178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.682676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 43/82] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:37.683719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.732110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c642b4-4a04-4dd6-b9bc-e399cade4ff9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.766234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.766797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.767077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.767347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.768488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.769947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.771539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 40/83] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:37.772304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.789142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8951-9aac-4bcb-8ed3-a87d3455aa55 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.822155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.822757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.823041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.823315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.847433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.874757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbd5fd-f4ae-453c-bb37-e554c5ec44ce None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:37.876318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:37.893691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:37.895504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:37.910130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:37.910768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:37.911051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:37.911326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:37.921602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:37.927568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:37.928458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:37.929239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 41/84] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:37.929886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:37.997129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 44/85] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:37.998673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.031435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d412461-e2b4-4f5f-b33c-06683bcb2372 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.051766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.064117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.064756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.065038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.065316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.066454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.067972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.069607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 42/86] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.070353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.096539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8cfec-8618-4495-8fb8-2207b3cafc6d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.101025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.129805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.130442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.130725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.131003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.132156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.133603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.135230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 45/87] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:37 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.136029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.173220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3bf4-3cbb-42e2-8fb4-630a23889b26 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.205878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.206488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.206765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.207038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.207711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.209151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.229928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 43/88] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:38 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:38.230539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.240086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6795222-5b15-4bc6-b092-c72934a6f518 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.267082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.282584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.283133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.283412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.283975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.284424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:38.284733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.286288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.296143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 46/89] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.296852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.349429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.368885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496219-b4bc-4c0b-80bd-6045cc118586 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.410783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.411441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.411802 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.411899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef373-b946-4548-9f5a-d676f9ae5db8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.412126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.413312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.414673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.417170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 44/90] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.417282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.465672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.466299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.466583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.467444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.468499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=6c7d397143574599b62017bc352e1183) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:38.468594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.470740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.478070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 47/91] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/role_assignments?user.id=6c7d397143574599b62017bc352e1183&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.479031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.556076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eee31-a66b-441a-90ba-dbe98713ad24 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.564826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.607022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.608573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.609073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.609854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.611078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:38.611872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.614528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.622483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 45/92] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.624482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.637318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a990e2c-e7ba-4a75-91b7-819410b5d6f7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.655740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.689909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.690604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.690890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.691171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.692994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.695196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.697276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 48/93] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.703472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:38.776225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005166bc-d444-439f-862f-bea9178b6237 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.818863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.820030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.820585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.821198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:38.822674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:38.823202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:38.825422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:38.842271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 46/94] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:38.891071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b363-fc8b-404a-9c52-5f48d464cb25 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:38.916224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.942720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:38.943356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:38.943668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:38.943941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:39.036630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:39.038201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:39.152205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 49/95] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:34:39.157828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:39.266524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:39.288394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd16688-f4ae-40f4-a224-6e548e75f984 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:39.323425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:39.324092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:39.324374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:39.324650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] PATH_INFO: `/v3/projects/ea9282c948354eca878c1d476de54c26` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:39.325791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] RBAC: Authorizing `identity:get_project(project_id=ea9282c948354eca878c1d476de54c26)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:39.327977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f23134-ff5a-4235-afc0-09fc5f4550ea None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:39.329799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 47/96] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:39 2022] GET /identity/v3/projects/ea9282c948354eca878c1d476de54c26 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:39.335578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:39.354988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:39.458658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9aac95-86f7-4462-a98f-ff754fae14e5 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:39.494204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:39.494823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:39.495103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:39.495378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:39.496131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=ea9282c948354eca878c1d476de54c26, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:39.496437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:39.497900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd817df-65e2-40a6-8b1a-61f92e6b3f2a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:39.503824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 50/97] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:39 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=ea9282c948354eca878c1d476de54c26&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:41.055807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074a0e-84da-46a2-9fdf-3cbd792a5744 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.056837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074a0e-84da-46a2-9fdf-3cbd792a5744 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.057326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074a0e-84da-46a2-9fdf-3cbd792a5744 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.059169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 48/98] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:41.065571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad32de4-1e49-4bfd-ad63-52edbc50cf9f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.066384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad32de4-1e49-4bfd-ad63-52edbc50cf9f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.066863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad32de4-1e49-4bfd-ad63-52edbc50cf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.085478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-bad32de4-1e49-4bfd-ad63-52edbc50cf9f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:41.086991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-bad32de4-1e49-4bfd-ad63-52edbc50cf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:41.157487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 51/99] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:41.285852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437660d-e6cc-43d9-a49a-c6c1c4452794 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.286497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437660d-e6cc-43d9-a49a-c6c1c4452794 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.287036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437660d-e6cc-43d9-a49a-c6c1c4452794 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.288630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 49/100] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:41.298050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.298614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.299089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.302793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f775d2-5bdc-40a9-b3a8-609fed5ade25 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.303699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f775d2-5bdc-40a9-b3a8-609fed5ade25 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.304737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f775d2-5bdc-40a9-b3a8-609fed5ade25 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.309004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 52/101] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:41.318527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a0ccc-0d2c-4be0-bf17-e8764ceec25b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.319091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a0ccc-0d2c-4be0-bf17-e8764ceec25b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.319592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a0ccc-0d2c-4be0-bf17-e8764ceec25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.338330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:41.339866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:41.354685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6c9a0ccc-0d2c-4be0-bf17-e8764ceec25b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:41.356196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6c9a0ccc-0d2c-4be0-bf17-e8764ceec25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:41.361671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:41.500428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 50/102] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:41.519444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 53/103] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:41.555840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:41.641388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ca91-b74b-4a69-905f-6e37984e4296 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.641619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ca91-b74b-4a69-905f-6e37984e4296 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.641927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985ca91-b74b-4a69-905f-6e37984e4296 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.643182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 54/104] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:41.659716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.659999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.660274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.674709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:41.689306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:41.690509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:41.705564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9554a-30b1-4d0b-9d3a-ab2c0ef8a5c7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:41.785442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:41.786098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:41.786380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:41.786653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:41.812364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:41.817215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:41.818068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:41.818847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 51/105] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:41.824384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:41.841063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 55/106] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:41.889512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:41.983262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.016576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d232b-9096-421e-a4fd-1d211a3ed493 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.055961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.056607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.056887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.057159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.059525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4dde-e4e5-48d0-9c7e-64efa43b6587 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.061800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.063261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.064900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 56/107] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:42.066637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.131676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.132853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.133262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.133543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.134214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:42.134920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.137208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.149161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 52/108] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:42.150612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.153360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.202247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc635d-ea6a-40fa-a2ce-955b36ec07d4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.237550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.238108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.238385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.238654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.245870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.250763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-abce2b03-7928-4107-ba64-6e73fa1e0eda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:42.255995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:42.256863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:42.257633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 57/109] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:42.258869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23797-550b-470a-b729-a56e787a8bcb None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.259149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23797-550b-470a-b729-a56e787a8bcb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.259429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23797-550b-470a-b729-a56e787a8bcb None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.261164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 58/110] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:42.266928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a23797-550b-470a-b729-a56e787a8bcb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.273093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6af271-969e-49d5-aeec-a8207a38bd11 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.307733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.311226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.311226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.311226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.311226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.311897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.340082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 53/111] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:34:42 2022] POST /identity/v3/projects => generated 302 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:42.340918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.396017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a23797-550b-470a-b729-a56e787a8bcb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.420592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.442832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.443602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.443951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.444291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] PATH_INFO: `/v3/users/4ca38bb072df4476b392c5fcba8c11f4` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.461475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] RBAC: Authorizing `identity:get_user(user_id=4ca38bb072df4476b392c5fcba8c11f4)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.463135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.464970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 59/112] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/users/4ca38bb072df4476b392c5fcba8c11f4 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:42.471132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.492958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402942c-5411-48e8-a297-3db36668a855 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.534829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.535152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.535388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.535742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.536913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.548775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-dc03fe81-be2d-4127-ae8f-8a7b094be96c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:42.553655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:42.554496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:42.555267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 54/113] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:42.556552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff457-7180-4a75-8216-f468ac18e628 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.557669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff457-7180-4a75-8216-f468ac18e628 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.557954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff457-7180-4a75-8216-f468ac18e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.575822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4d6ff457-7180-4a75-8216-f468ac18e628 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:42.577580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4d6ff457-7180-4a75-8216-f468ac18e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:42.593027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3cfdd-41d9-4f5f-9ad6-6c99ddef4cba None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.631395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.632067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.632355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.632620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.633290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:42.633594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.634975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.644416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 60/114] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:42.645025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 55/115] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07bab8-49a8-400a-8fa7-b513227b40b6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07bab8-49a8-400a-8fa7-b513227b40b6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07bab8-49a8-400a-8fa7-b513227b40b6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 56/116] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:42.663515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07bab8-49a8-400a-8fa7-b513227b40b6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.736100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.761348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90401dd-200e-4f17-b5ff-59d0c42e70f8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.781038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:42.801893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.802568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.802925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.803253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.812638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07bab8-49a8-400a-8fa7-b513227b40b6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.856083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:42.856657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:42.857003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:42.857394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:42.858631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:42.859185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.860460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.869973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 57/117] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:42.875159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:42.893583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:42.895690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2054f-e76f-40aa-9940-81c8b386a29f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:42.979424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c7501-b5bc-4e35-ae0c-7006f3d31d89 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:42.988698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.018318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.018878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.019155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.019425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.024431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 61/118] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:43.024431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.024431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.024431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.025057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.026501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.028278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 58/119] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:42 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.033810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.040826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:43.042035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:43.056604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.115327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 62/120] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:43.116124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.156546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3d8f5-fee2-49fd-aea0-f5f1c0877e2f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.203908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.205203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.205621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.206027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.207310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.209666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.212069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 59/121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.213298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.221791 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b6cf13-b6a2-4bc8-bdda-82b75bea2b02 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.259513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.260378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.260759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.261702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.272077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.273499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:43.278579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:43.280125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:43.281088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:43.283064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 63/122] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:43.288729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.303732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.343463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1655de0-f6da-4375-bee6-e867f7792478 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.382144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.382856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.383140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.383417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.384624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.386322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.388214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 60/123] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.389110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.424421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331be0e-7c59-4f1d-b5d8-14d42a961ce0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.459511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.460150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.460425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.460694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.461812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.463257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.470044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 64/124] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.470044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.508317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18146203-fc57-4a17-a04b-462a5772002a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.522184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.547397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.548192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.548544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.548877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.550046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.551579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.558156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 61/125] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.558156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.562069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.588889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec7157-4d39-446c-b5d1-95c978efac06 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.625776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.626375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.626654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.626922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.627601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:43.627901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.629261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.637161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 65/126] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.637836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.668877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874df53-7465-4b98-8486-64de3fe54f11 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.706979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.707653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.707934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.708204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.718424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:43.723145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:43.724147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 62/127] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:43.728306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.756165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1b39d-a2d1-4fe1-bdca-1e1defff8137 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.772842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.793216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.794133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.794412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.794684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.796022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=4ca38bb072df4476b392c5fcba8c11f4) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:43.796353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.797796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.803201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 66/128] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/role_assignments?user.id=4ca38bb072df4476b392c5fcba8c11f4&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=3abef7f219834a45b83613fe1f60cedd => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:43.803809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.813563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:43.843605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d61b26-e300-4ac7-83fc-afb27298391a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.883311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.884101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.884383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.884840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/9c01f65b4421415cbf105fa576ebed73/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.886972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, user_id=9c01f65b4421415cbf105fa576ebed73, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.888597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:43.905207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 63/129] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:43 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/9c01f65b4421415cbf105fa576ebed73/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:43.911198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:43.925165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c225-eb2b-40e0-a684-f53da1f8a74c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:43.985506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:43.987010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:43.987379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:43.987786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/6c7d397143574599b62017bc352e1183/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:43.990136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, user_id=6c7d397143574599b62017bc352e1183, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:43.991901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.026528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 67/130] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:43 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/6c7d397143574599b62017bc352e1183/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:44.027434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.063987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:44.070813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cec316-4a67-4285-bc1c-20ce2cb8bc72 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.115672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.116873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.117314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.117744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.119211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.122346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.125423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 64/131] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:43 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:44.136362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.182064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:44.267812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4c3a1-3d18-451b-a087-85ab57e33b9f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.319573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.320124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.320407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.320683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.321357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:44.321662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.323046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.332060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59104044-1b45-41e3-bda6-9bb294735bb7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.332882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 68/132] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:44.338584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.384243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.384916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.385204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.385484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.386718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.388247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.390851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 65/133] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:44 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:44.396524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.440035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:44.450419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470ceaa-bbaa-4cf9-9625-d6848a99d926 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.489100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:44.512777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.513423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.514272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.514726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.518462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.518462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b68431-4a86-47e4-a9c9-86524c77426f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.520025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 69/134] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:44 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:44.521289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3109c-4274-4fde-9641-24bb061d529e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.521289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3109c-4274-4fde-9641-24bb061d529e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.521289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3109c-4274-4fde-9641-24bb061d529e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.525071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 70/135] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:44.526145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3109c-4274-4fde-9641-24bb061d529e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.565596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f7eba-271a-4c06-b6ac-da43f60f7afd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.633467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.634054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.634332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.634603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/9c01f65b4421415cbf105fa576ebed73/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.639348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, user_id=9c01f65b4421415cbf105fa576ebed73, role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.640841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818313db-5ecf-4bd8-a875-fadc5028d16e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.663797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 66/136] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:44 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/9c01f65b4421415cbf105fa576ebed73/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:44.665090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.665373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.666247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.695092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:44.696368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:44.715878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3109c-4274-4fde-9641-24bb061d529e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.755720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.756377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.756661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.756939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] PATH_INFO: `/v3/projects/ea9282c948354eca878c1d476de54c26` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.758690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] RBAC: Authorizing `identity:get_project(project_id=ea9282c948354eca878c1d476de54c26)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.760207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:44.762354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 71/137] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:44 2022] GET /identity/v3/projects/ea9282c948354eca878c1d476de54c26 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:44.771838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:44.834028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 67/138] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:44.885290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:44.931661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c21795-5811-4dbd-b698-7d37170d7868 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:44.978092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:44.978766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:44.979045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:44.979367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] PATH_INFO: `/v3/projects/ea9282c948354eca878c1d476de54c26/users/9c01f65b4421415cbf105fa576ebed73/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:44.981333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea9282c948354eca878c1d476de54c26, user_id=9c01f65b4421415cbf105fa576ebed73, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:44.983370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:45.006461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 72/139] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:44 2022] PUT /identity/v3/projects/ea9282c948354eca878c1d476de54c26/users/9c01f65b4421415cbf105fa576ebed73/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:34:45.206324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:45.241896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:45.344911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31036ac5-7fb6-4925-8b1c-7ea469d855ed None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:45.389329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:45.390329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:45.390831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:45.391301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:45.392672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:45.394339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755af6e-58bf-477e-a436-9886b55b8e78 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:45.397443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 68/140] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:45.404663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:45.536049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc6124-2176-4357-abda-a8909b39e7a0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:45.563097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:45.584219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:45.584838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:45.585115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:45.585393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:45.586113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:45.587592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86526e4e-5392-4ed4-967e-fc3778c00493 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:45.627000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 73/141] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:45 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:46.508439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3188f-303c-4a26-8ebb-b3f7235dd6de None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:46.508934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3188f-303c-4a26-8ebb-b3f7235dd6de None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:46.509005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3188f-303c-4a26-8ebb-b3f7235dd6de None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:46.510619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 69/142] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:46.518465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:46.518465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:46.518465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:46.544544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:46.545813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:46.636320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffa1-53ab-41d5-8ba7-48a9d2d5c9d4 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:46.636653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffa1-53ab-41d5-8ba7-48a9d2d5c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:46.636879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cffa1-53ab-41d5-8ba7-48a9d2d5c9d4 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:46.638235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 70/143] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:46.644413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:46.644693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:46.644971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:46.655634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 74/144] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:46.668503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:46.669728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:46.747291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 71/145] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:47.180807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.294108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.303779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d568e3f-1e0b-4c78-b061-71a9985f2b4d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.317886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:47.334260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:47.400542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:47.401292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.401577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.401846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.428240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:47.435553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:47.436402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:47.437233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 75/146] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:47.444504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.454437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e05f7-a816-4993-8f56-5b6623c5fa62 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.507093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:47.507927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.508217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.508490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.525732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8c94a1cf-119b-4227-805f-77a47707a279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:47.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:47.535323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 72/147] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:34:47.536146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ec4ee-ecf3-4108-b921-a6b69ff58b64 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.536830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ec4ee-ecf3-4108-b921-a6b69ff58b64 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.536830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ec4ee-ecf3-4108-b921-a6b69ff58b64 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.543117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 73/148] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:47.543973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ec4ee-ecf3-4108-b921-a6b69ff58b64 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.640766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6fe3d-0e36-40a1-847c-d2d4c87247b9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.659054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:47.676960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:47.677584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.677866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.678146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.678820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:47.679123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:47.681424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1317d1-69f4-4112-9710-e0909a82b2ec None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:47.681773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ec4ee-ecf3-4108-b921-a6b69ff58b64 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.692732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 76/149] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:47.694318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.694425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.695535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.715097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:47.716390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:47.721546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:47.722160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.722433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.722696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.723358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:47.723686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:47.725184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255e992-7b3e-497f-b354-2bfd84c3c10c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:47.735434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 74/150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:47.737139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058a66-6a12-4950-a430-0e6773dd0878 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.737426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058a66-6a12-4950-a430-0e6773dd0878 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.737797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17058a66-6a12-4950-a430-0e6773dd0878 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.739221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 75/151] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:47.740018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17058a66-6a12-4950-a430-0e6773dd0878 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.749242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:47.796407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 77/152] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:47.797465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:47.854109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17058a66-6a12-4950-a430-0e6773dd0878 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.889425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:47.890065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.890342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.890612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] PATH_INFO: `/v3/users/4ca38bb072df4476b392c5fcba8c11f4` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.891902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] RBAC: Authorizing `identity:get_user(user_id=4ca38bb072df4476b392c5fcba8c11f4)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:47.893367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bb5ed-2259-4c64-9e91-cd679412bc69 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:47.895107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 76/153] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/users/4ca38bb072df4476b392c5fcba8c11f4 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:47.896094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca597cd-ea0c-4cb1-83b2-0f4b72162af6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.896373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca597cd-ea0c-4cb1-83b2-0f4b72162af6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.896648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca597cd-ea0c-4cb1-83b2-0f4b72162af6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.898153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 77/154] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:47.899261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:47.899595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:47.899942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:47.920638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:47.921823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:47.938357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c37ca-52cb-4812-b770-d11a816c8010 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:47.944029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.004876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.005522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.006290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.006568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] PATH_INFO: `/v3/users/6c7d397143574599b62017bc352e1183` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.008858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] RBAC: Authorizing `identity:get_user(user_id=6c7d397143574599b62017bc352e1183)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:48.009040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 78/155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:48.009902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.014329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1ed9b-423f-400f-92e6-944eacaf5611 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:48.019471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 78/156] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:47 2022] GET /identity/v3/users/6c7d397143574599b62017bc352e1183 => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:48.020760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.021036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.021302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.060153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:48.061990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:48.085399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.168157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 79/157] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:48.169317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.259270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7dc2d-695b-4b5e-929e-4280d7977cbc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.332110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd868a-7c57-4300-986c-d453015e605e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.335532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.336181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.336454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.336722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.337846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:48.339818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:48.342624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 79/158] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:48.348033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.378028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.408753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.409442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.409645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.409914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.459257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.530917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:48.532428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:48.608041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527507-1e8d-49f6-a662-f6d4afe47608 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.675241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.676386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.676663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.676929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/4ca38bb072df4476b392c5fcba8c11f4/roles/3abef7f219834a45b83613fe1f60cedd` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.679813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, user_id=4ca38bb072df4476b392c5fcba8c11f4, role_id=3abef7f219834a45b83613fe1f60cedd)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:48.681270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:48.706998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 80/159] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:48 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/users/4ca38bb072df4476b392c5fcba8c11f4/roles/3abef7f219834a45b83613fe1f60cedd => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:48.707854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.714536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 80/160] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:48.716587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.808973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.831294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:48.880700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cda71-6cd3-4fe3-97e6-0c961484a3a5 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.909689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fbbb1-39fb-41c1-86ef-e2eb69470858 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:48.918891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.919559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.919879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.920152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:48.932357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:48.938488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:48.939541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:48.940358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:48.943617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 81/161] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:48.944559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:48.988163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:48.988813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:48.989079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:48.989347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.004235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:49.013206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:49.014255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:49.015035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 81/162] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:49.016206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.060638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4889e-7c32-4ee2-8e82-1cb7638fb851 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.080698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.109846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.110493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.111079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.111355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.113461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.114927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.117054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 82/163] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:48 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.118034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.194760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d05c35-5ab7-4aba-89c1-fcaa856648d2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.254505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.266358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.267007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.267282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.267580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.279111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:49.282827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612005-c656-45c1-8eeb-e461470ec679 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:49.284641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:49.285712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:49.286487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 82/164] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:49.287170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.354011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.354765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.355182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.355614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.356522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:49.356978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.359110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.370286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 83/165] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.376205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.395054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb863a03-5dae-4d5f-b4a2-2214f92dd1e4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.427923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.430622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.431747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.431747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.431938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.432465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:49.432764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.434225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.443757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 83/166] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.444406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.505110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.538343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d7ec-a901-4d96-89bd-d94b7bd3bcee None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.555949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81247f9b-f506-438d-ae43-be0617c41c75 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.591330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.592649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.592990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.593324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.594056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:49.594421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.595911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.604729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 84/167] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.605583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.609707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.610331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.610615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.611209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.611976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=6c7d397143574599b62017bc352e1183) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:49.612282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.613732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.631211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 84/168] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/role_assignments?user.id=6c7d397143574599b62017bc352e1183&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.632251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.711987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c9e0-3295-45b7-807a-c52b9fa61690 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.726273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.747079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.747800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.748659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.748933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.750079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.751559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.753244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 85/169] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.754066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.865526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bbfc8-692f-49d5-9698-f6a08bf50272 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.888259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ced32e-109e-41bb-8256-aa4065e774c5 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:49.902751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.903398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.903706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.903976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.905715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.906232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.907208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.909076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 86/170] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.909899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:49.954152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:49.954881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.955160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.955454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] PATH_INFO: `/v3/users/9c01f65b4421415cbf105fa576ebed73` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.960289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] RBAC: Authorizing `identity:get_user(user_id=9c01f65b4421415cbf105fa576ebed73)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:49.961766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe355057-434e-4ab9-ba3b-769a565776ec None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:49.963643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 85/171] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/users/9c01f65b4421415cbf105fa576ebed73 => generated 275 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:49.964048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:49.966095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58758d13-0986-4419-982d-8d58ee148bbb None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:49.966095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58758d13-0986-4419-982d-8d58ee148bbb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:49.966095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58758d13-0986-4419-982d-8d58ee148bbb None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:49.967810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 86/172] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:49.968321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58758d13-0986-4419-982d-8d58ee148bbb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.029414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b29a7-01da-4851-97cd-d174af9220de None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.065418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.066088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.066365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.066630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.140457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:50.141993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117cc04-beea-41c3-9eda-34cfe5762a09 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:50.148195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58758d13-0986-4419-982d-8d58ee148bbb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.196900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.213530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.214163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.214449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.214727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.250948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 87/173] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:50.252263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.252555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.253189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.272569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:50.273795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:50.347307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 88/174] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:50.348205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.391929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.457553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:50.459027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:50.513693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.557236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a88d-81b4-4038-8f5a-1f10e1b915fc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.610522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.612061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.612517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.612944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.631716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 87/175] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:50.632890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.707910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:50.709324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:50.719779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.746231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baa5ab-610b-4cf8-9c62-89d6e8b366c7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.783274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.784236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.784787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.785194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.786590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:50.788201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14e1a-1313-4c02-9bea-d0a376d5028d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:50.790171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 88/176] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:50 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:50.791424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd09c70-ebfc-450b-95fd-7ae22236559f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.791926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd09c70-ebfc-450b-95fd-7ae22236559f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.792334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd09c70-ebfc-450b-95fd-7ae22236559f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.793921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 89/177] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:50.795836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63bcc1-eedf-44b0-ab9a-6e8ca30e28ab None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.796250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63bcc1-eedf-44b0-ab9a-6e8ca30e28ab None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.796677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63bcc1-eedf-44b0-ab9a-6e8ca30e28ab None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.798156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 90/178] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:50.799001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63bcc1-eedf-44b0-ab9a-6e8ca30e28ab None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.808364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.830459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 89/179] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:50.831412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.912139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63bcc1-eedf-44b0-ab9a-6e8ca30e28ab None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.944907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c72bd0-17f4-4a80-99f7-bf54f973cf2e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:50.947978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.948110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.948367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.948629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] PATH_INFO: `/v3/projects/ea9282c948354eca878c1d476de54c26` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.949729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] RBAC: Authorizing `identity:get_project(project_id=ea9282c948354eca878c1d476de54c26)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:50.951755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:50.953575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 91/180] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:50 2022] GET /identity/v3/projects/ea9282c948354eca878c1d476de54c26 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:50.954468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:50.971386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:50.987472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:50.988127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:50.988404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:50.988778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:50.999449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4d8b1446-d743-444b-a4e5-254a523dabe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:51.004211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:51.005629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 90/181] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:51.006868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb56bc-cd44-4c32-b9cb-c6d441059038 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.007644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb56bc-cd44-4c32-b9cb-c6d441059038 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.007644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb56bc-cd44-4c32-b9cb-c6d441059038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.025313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7aeb56bc-cd44-4c32-b9cb-c6d441059038 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:51.026501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7aeb56bc-cd44-4c32-b9cb-c6d441059038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:51.062499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.067147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2cb98-1867-4d74-8fea-26530069a74b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.099776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 91/182] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:51.103378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.103378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.103378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.117389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.118333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.118834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.119262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.119766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:51.120335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:51.120839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.121234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:51.122648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.137380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 92/183] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:50 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.137380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.243577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 92/184] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:51.244577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.300779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.339129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5044-21b0-468e-9a40-4583de0c8bb2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.411770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.412414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.412690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.412965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.414216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.415681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.422169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 93/185] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.422965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.439900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.443106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9c72-5dbd-4693-a977-8084f568cd44 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.500164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.500695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.500971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.501240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.501962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=ea9282c948354eca878c1d476de54c26, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:51.502264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.503842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.510631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 93/186] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=ea9282c948354eca878c1d476de54c26&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.511223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.582880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17080d-8c05-49eb-af4e-bb768b646b5e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.630400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.631033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.631314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.631620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.632299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:51.632601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.634081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.647357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 94/187] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.647962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.661501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.697329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e97685-2e8e-4105-ac75-6f55d0dadaac None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.738463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.739077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.739807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.740110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.740846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=9c01f65b4421415cbf105fa576ebed73) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:51.741159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.743359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.749893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 94/188] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/role_assignments?user.id=9c01f65b4421415cbf105fa576ebed73&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.750765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.776653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d207da7-512c-4231-bbd4-79e5728eced4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.785400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.821188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.821844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.822119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.822383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.835526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:51.842852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:51.843839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:51.844625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 95/189] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:51.846000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.882341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fbe9a-ad5a-4ba0-b6e8-3b1a2cb357a7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:51.921299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:51.922397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:51.922832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.923013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.923240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.925725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:51.928026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897db6c5-1c75-43e9-b15b-6deeb54cf744 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:51.929677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 95/190] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:51.930734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4656c-dd39-45b9-906e-972e6de98591 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:51.931022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4656c-dd39-45b9-906e-972e6de98591 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:51.931295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4656c-dd39-45b9-906e-972e6de98591 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:51.933152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 96/191] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:51.933801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4656c-dd39-45b9-906e-972e6de98591 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:51.966362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47db757-6823-424b-b636-4588ea6d8ed6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.005642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.006205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.006484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.006749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.020263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.021745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.023534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 96/192] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:52.024124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.039128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.048281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4656c-dd39-45b9-906e-972e6de98591 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.081838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.082447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.082721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.082987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.083665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:52.083960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.085322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf24b8-3470-4df5-82b0-2723f43842fd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.094302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 97/193] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:52.095668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.096128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.096603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.114970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:52.116185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:52.120954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf3d7c-5667-462f-8df5-958370aebe0b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.145053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.153706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.154948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.155217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.155495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.156165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.157591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.178440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 97/194] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:52 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:52.179243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.187337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 98/195] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:52.192684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.308974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1c4b-69e9-445f-bf07-4d51bf364f79 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.314156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.338027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a6476-6047-47da-9f0a-e496d540e335 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.355472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.356341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.356737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.357013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.386132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.387415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.388006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.388457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.390819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.393438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.396307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 99/196] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:52.405229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.442049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.465006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.466626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.540337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f683-416c-4580-b186-6d4c1d146ac6 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.584541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.585248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.585624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.585987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.604310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 98/197] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:52.607498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.620810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.678016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.679512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.712234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.738668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ab6f-6f70-449c-8f4f-071f7be6a2a3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.790466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 100/198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:34:52.791516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.792294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.792937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.793213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.793478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.808146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:52.814814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:52.815702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:52.817123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 99/199] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:52.820162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.859669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.918314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a724003-6319-4900-9aad-5e15f751b565 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.939189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2fcac-d606-48c0-9422-8e3300ff89f8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.967613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.968519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.968804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.969536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.970952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.972458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.974131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 101/200] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:52.974925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.980066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.980974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.981449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.981925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.983965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.986411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.989160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 100/201] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:52.990279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.006902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:53.104225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca22f-dadb-447d-a50d-a2246a3a7dd4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.116838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fef4-747d-4753-aac3-7908540ef4a1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.143172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:53.150482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:53.152132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.152588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.153128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.154031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=53152c4c72a940d0ad29a99e40bbf6aa) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:53.155067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:53.156471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:53.157028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:53.157133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.157505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.157860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.158582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:53.158950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:53.160481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:53.164577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 101/202] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/role_assignments?user.id=53152c4c72a940d0ad29a99e40bbf6aa&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:53.165534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.169974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 102/203] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:53.180870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.261977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:53.293781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0d9b-0d53-418c-bcbc-64e84abf4f9b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.336520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:53.337536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.338015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.338496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.339449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=53152c4c72a940d0ad29a99e40bbf6aa) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:53.340003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:53.342959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:53.351235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 102/204] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:53 2022] GET /identity/v3/role_assignments?user.id=53152c4c72a940d0ad29a99e40bbf6aa&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:53.391757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2efc-86e6-4212-a2c4-56d51e44b91c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.439075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:53.439748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.440026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.440295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] PATH_INFO: `/v3/users/4ca38bb072df4476b392c5fcba8c11f4` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.441515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] RBAC: Authorizing `identity:get_user(user_id=4ca38bb072df4476b392c5fcba8c11f4)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:53.442962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:53.444622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 103/205] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:53 2022] GET /identity/v3/users/4ca38bb072df4476b392c5fcba8c11f4 => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:53.450474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.598927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24677e8-32d9-4cfe-8e47-e1c63c178b4c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:53.648424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:53.648967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:53.649247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:53.649580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:53.658803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:53.739080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:53.740645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:53.852038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 103/206] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:53.858150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:53.958538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:54.037660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8457bfa-bc2f-454e-a1d2-fd2312c49b1a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:54.096333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:54.096975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.097256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.097525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.102910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:54.104397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2a501-4989-4133-b813-aeb95f6a3e42 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:54.106268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 104/207] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:53 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:54.112905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:54.170734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:54.243737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e2c75-7669-4ba8-afae-9d2dbbba2760 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:54.287577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:54.288193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.288468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.288733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.289455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, scope.project.id=139d28049ff14e92b984e2e272c11ae8, user.id=4ca38bb072df4476b392c5fcba8c11f4) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:54.289763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:54.291212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e575bf-18d6-44da-ae66-3fa3f38e66ea None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:54.298094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 104/208] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:54 2022] GET /identity/v3/role_assignments?user.id=4ca38bb072df4476b392c5fcba8c11f4&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=3abef7f219834a45b83613fe1f60cedd => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:54.587009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55acd6-e6b7-4edc-8440-1f1ded46be31 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.587323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55acd6-e6b7-4edc-8440-1f1ded46be31 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.587585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55acd6-e6b7-4edc-8440-1f1ded46be31 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.589551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 105/209] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:54.593868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601357-f63a-4d89-95cf-5f0ef44d5076 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.594222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601357-f63a-4d89-95cf-5f0ef44d5076 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.594441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca601357-f63a-4d89-95cf-5f0ef44d5076 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.597327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 105/210] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:54.597435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.597522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.597797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.602513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:54.602794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:54.603063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:54.620648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:54.621987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:54.626090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:54.627941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:54.701524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 106/211] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:54.726348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 106/212] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:54.985000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.008964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.042273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.052998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.111702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480740d5-a339-49ae-8cbf-c521a150ca30 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.128476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22295899-142e-4739-9c5c-f360b51a2255 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.159280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.160332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.160872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.161147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.164587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.165219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.165499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.165769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.166960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:55.168489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:55.170978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 107/213] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:55.176437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.177277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:55.182218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:55.183099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:55.184271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 107/214] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:55.189250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.298969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d6a63-7b5e-4928-9c32-cdcc030499dd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.312871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.316760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.321636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d999ab-67cd-4b40-b245-f2eed572f17b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.338425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.339111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.339464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.339836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.340570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:55.341035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:55.343177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:55.356801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 108/215] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:55.357930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.366598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.367543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.367991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.368900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.369743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:55.371852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a9c21-8d57-4943-a749-8c1b423c9de2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:55.407503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 108/216] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:55 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:34:55.483921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be84d-199f-4e23-a2d3-86fe1b29b14f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.517352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4891ab-4115-48d0-8911-8fc9921fa6e2 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.517685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4891ab-4115-48d0-8911-8fc9921fa6e2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.517846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4891ab-4115-48d0-8911-8fc9921fa6e2 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.519211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 109/217] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:34:55.526866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b5bc9-4c9f-49e3-ad05-c78edf46f69e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.527150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b5bc9-4c9f-49e3-ad05-c78edf46f69e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.527426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b5bc9-4c9f-49e3-ad05-c78edf46f69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.530942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.530942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.530942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.531211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] PATH_INFO: `/v3/users/a24bbf548bf440fe9e64df14d7f97ed5` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.546872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] RBAC: Authorizing `identity:get_user(user_id=a24bbf548bf440fe9e64df14d7f97ed5)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:55.548405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:55.550579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 109/218] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/users/a24bbf548bf440fe9e64df14d7f97ed5 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:55.554484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a45b5bc9-4c9f-49e3-ad05-c78edf46f69e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:55.555810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a45b5bc9-4c9f-49e3-ad05-c78edf46f69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:55.556507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.597310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.647767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 110/219] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:55.656962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c4b84-1f0d-46ca-b2f6-6d0418e3ffa0 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.657267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c4b84-1f0d-46ca-b2f6-6d0418e3ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.657505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c4b84-1f0d-46ca-b2f6-6d0418e3ffa0 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.658845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 111/220] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:55.664146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.664589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.665001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.685107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:55.687053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:55.693545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427282e-aa5a-40bc-b406-b477ef5e1047 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:55.736920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:55.737543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:55.737821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:55.738089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:55.778831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 112/221] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:55.827803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:55.829202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:55.862486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:55.937325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 110/222] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:55.943221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.944696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:55.965689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.075412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98f480-333e-4984-85f2-471428cc2109 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.078684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bccd2-c678-4feb-a055-d18e3edc1e03 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.111915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.112564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.112841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.113122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.114484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.115975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.117779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 113/223] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:56.120399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.124582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.133584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.134536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.135040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.135542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.161285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:56.167140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:56.168069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:34:56.168852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:56.170466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 111/224] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:56.176482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.236153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.244374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6081e2-8da9-484c-9571-17784738243d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.283037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aec1fa-acf5-4d6c-87e2-efa6969b852e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.284814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.285511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.285786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.286050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.287305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=a24bbf548bf440fe9e64df14d7f97ed5) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:56.287305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.288618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.294687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 114/225] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/role_assignments?user.id=a24bbf548bf440fe9e64df14d7f97ed5&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:56.295756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.322865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.323942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.324440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.324920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.337707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:56.343552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:56.344477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:34:56.345278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:34:56.347741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 112/226] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:56.361294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.422085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe819515-06e5-4d6e-ad3a-d765df559629 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.469812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.470443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.470775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.471049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.471760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:56.472060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.474759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.487312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.491103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 115/227] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:56.497775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.527583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.553068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0c47a-b829-4304-a99f-ad423d04d827 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.599207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.599766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.600040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.600306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.600968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:56.601471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.604773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.619820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 113/228] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:56.622033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.729703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66fde4-7084-4771-8ca5-9671317a20db None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.738701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc95ede-fd57-4a99-a1be-8d9a6e88709e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.759107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:56.778017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.778672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.778954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.779274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.780473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.781954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.784010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 114/229] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:56.789975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.809860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.811165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.811442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.811784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.812942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.814506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.816187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 116/230] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:56.823619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:56.906836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3affc-2e7e-4574-8404-b9de841d69f1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:56.944674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:56.945328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:56.945612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:56.945893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:56.947090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:56.948922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:56.950978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 115/231] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:56.957700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:57.007578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:57.034638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c5820-41f4-4fed-9508-3e2d840ffa3f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.070982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.071658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.071939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.072216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.073368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.074887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.077520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 117/232] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:56 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:57.082528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:57.083304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:57.183560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d09351-d1a2-485d-a09a-74edd6d3f650 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.187849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65001039-e1f7-41e2-a6c4-c31bdd5abe01 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.242096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.242845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.243044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.243323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/users/53152c4c72a940d0ad29a99e40bbf6aa/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.246544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, user_id=53152c4c72a940d0ad29a99e40bbf6aa, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.248563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcd4ac-fd64-4ada-a0fe-6f8c76038764 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.253318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.253982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.254262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.254535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/users/53152c4c72a940d0ad29a99e40bbf6aa/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.258368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, user_id=53152c4c72a940d0ad29a99e40bbf6aa, role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.260154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769cd16-8501-48c7-9413-98c7fda3d248 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.274371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 118/233] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:57 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/users/53152c4c72a940d0ad29a99e40bbf6aa/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:57.302940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 116/234] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:34:56 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/users/53152c4c72a940d0ad29a99e40bbf6aa/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:34:57.740646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809ac52-9e74-462a-9b2f-a4f0eb8a0b84 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.741344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809ac52-9e74-462a-9b2f-a4f0eb8a0b84 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.741835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809ac52-9e74-462a-9b2f-a4f0eb8a0b84 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.743413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 119/235] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:57.749586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa664f8-6bd1-4d7b-b377-86f3cc839911 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.750163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa664f8-6bd1-4d7b-b377-86f3cc839911 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.750640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa664f8-6bd1-4d7b-b377-86f3cc839911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.770304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-baa664f8-6bd1-4d7b-b377-86f3cc839911 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:57.771843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-baa664f8-6bd1-4d7b-b377-86f3cc839911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:57.869441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 117/236] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:34:58.072767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ae8fc-afe4-44fa-b637-1ef8033b4915 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.073419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ae8fc-afe4-44fa-b637-1ef8033b4915 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.073920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ae8fc-afe4-44fa-b637-1ef8033b4915 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.075780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 120/237] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:58.081517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.081800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.082073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.105062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:58.106238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:58.153001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:58.195837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 118/238] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:58.251774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ae8fc-afe4-44fa-b637-1ef8033b4915 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.262138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:58.363944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ae8fc-afe4-44fa-b637-1ef8033b4915 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.400050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.401079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.401578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.402040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.413412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:58.418815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:58.419720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:34:58.421074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:58.423366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 121/239] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:58.429285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.567978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0cd89-25a2-4538-a634-01f0a3125dd2 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.595979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.619062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.619629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.619909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.620179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.627036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] RBAC: Adding query filter params (name=reader) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:58.627036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.627036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b9ffd-3b1f-4c0b-b93a-b9a1e3079efb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.632014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 119/240] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:58.637045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946eb1e-734c-43ab-bfa2-62027ea7e64d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.637337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946eb1e-734c-43ab-bfa2-62027ea7e64d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.637610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946eb1e-734c-43ab-bfa2-62027ea7e64d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.639775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 120/241] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:58.640417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946eb1e-734c-43ab-bfa2-62027ea7e64d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.677430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:58.681174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:58.725078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb685b2-543e-4ab1-b7b6-2201eb2face6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.761905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946eb1e-734c-43ab-bfa2-62027ea7e64d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.776324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.777556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.778426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.779239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.780909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.782915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62605ab-d135-4792-8529-48ca2bc6af23 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.785599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 122/242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:58.786909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.787408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.787962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.804248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.804604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.804879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.805193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] PATH_INFO: `/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.811870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:58.813290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:58.820551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca2e6a88d6b48a0803bf97cf5f5c03f)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.822000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.823937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 121/243] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:58.824623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.902720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 123/244] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:58.903913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.938724 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:58.963541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0914219-56d8-4dfb-b09a-de63c4bc23f6 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.991844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.019282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35af7e-d09b-4aad-9c75-e95d27e809fa None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.058181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.059583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.059999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.060420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.082589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.083282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.083648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.084286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.084729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.086285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.135499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.137540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.139745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 122/245] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:59.207804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.259469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 124/246] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:34:59.266279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.274249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.400959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4ea96-db0e-48b7-83a5-13980ed903ce None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.445340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501e449-80f9-49b4-8816-de83860cbe1c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.451938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.452580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.452855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.453126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.466647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:34:59.472709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:34:59.473606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:34:59.474493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 125/247] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:34:59.479883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.492324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.493462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.493982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.494467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.496087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.497818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.500508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 123/248] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:59.506312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.510286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.531874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.630413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e01df-89da-4811-8b4b-d5f129f0c0bb None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.651915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa884646-7189-46c3-9c01-70d60764d11f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.677663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.677663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.677663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.677663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.678251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:59.678570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.679962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b4ce3-c442-41b3-addf-5c09be8d4d44 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.692622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 126/249] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:59.694644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288819b2-92b7-4e02-b229-84d02defdacc None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.694931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288819b2-92b7-4e02-b229-84d02defdacc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.695362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288819b2-92b7-4e02-b229-84d02defdacc None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.697047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 127/250] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:59.699398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288819b2-92b7-4e02-b229-84d02defdacc None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.711508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.712133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.712718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.713001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.714028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=5ca2e6a88d6b48a0803bf97cf5f5c03f) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:59.714339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.716506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac2f58-fec0-4d51-9b0f-9d53d848bf14 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.725253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 124/251] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/role_assignments?user.id=5ca2e6a88d6b48a0803bf97cf5f5c03f&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=3abef7f219834a45b83613fe1f60cedd => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:59.726962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.727295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.727921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.752749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:59.753976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:59.820493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.845385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288819b2-92b7-4e02-b229-84d02defdacc None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.853846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 125/252] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:34:59.895050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.895727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.896007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.896278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] PATH_INFO: `/v3/users/a24bbf548bf440fe9e64df14d7f97ed5` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.897413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] RBAC: Authorizing `identity:get_user(user_id=a24bbf548bf440fe9e64df14d7f97ed5)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.898858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0b3b-5587-4a08-a4b9-c1b08399aff1 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.900485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 128/253] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/users/a24bbf548bf440fe9e64df14d7f97ed5 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:34:59.906775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.924328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.967794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3375a-68ad-443a-9ae0-b62a244b497a None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.968120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3375a-68ad-443a-9ae0-b62a244b497a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.968343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3375a-68ad-443a-9ae0-b62a244b497a None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.969702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 129/254] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:34:59.975070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.975405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.975680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.003889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:00.005365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:00.037218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6834f8-48c6-4be0-ac48-24ba2ca15fc6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.075782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.077118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.077395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.077944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.080098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.081613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.083504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 126/255] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:00.089424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.149843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 130/256] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:00.150705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.219948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.251404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ac3e5-72e0-4ec0-8646-dc780a372eae None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.290096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.291148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.291645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.292104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/users/a24bbf548bf440fe9e64df14d7f97ed5/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.294212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, user_id=a24bbf548bf440fe9e64df14d7f97ed5, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.296435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.321515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 127/257] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:00 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/users/a24bbf548bf440fe9e64df14d7f97ed5/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:00.321695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.344630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3592d1d-e06d-43d2-a6cd-8b352e2f927d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.410797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.411463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.411769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.412037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.427426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:00.432235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:00.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:00.434315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 131/258] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:00.440147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.557960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.652716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf588d9-24f2-4762-b446-9cbc421b10c3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.727708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caedff-d67b-4ccf-8734-2e8d54302a9c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.728773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.729408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.729687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.729957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.730636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:00.730946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.732897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.743835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.751828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 128/259] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:00.757041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.767266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.767935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.768211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.768480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.779507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:00.784547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:00.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:00.786770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 132/260] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:00.792245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.796424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.903376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0946-ff43-41af-8e37-067c3b2687a6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.939927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.940688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.940970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.941239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.941897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:00.942194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.943605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.952911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 133/261] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:00.958939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:01.031823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:01.066869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da2f7d-2604-4b73-b774-7186e50b52f3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:01.067757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd3851-c821-4871-8082-fb0e7b5c3dbc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:01.103753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:01.104390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:01.104664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:01.104933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.106075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.108112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.110313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 134/262] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:01.116591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:01.156223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:01.156887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:01.157168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:01.157441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] PATH_INFO: `/v3/users/53152c4c72a940d0ad29a99e40bbf6aa` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.158649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] RBAC: Authorizing `identity:get_user(user_id=53152c4c72a940d0ad29a99e40bbf6aa)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.160158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.162087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 129/263] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/users/53152c4c72a940d0ad29a99e40bbf6aa => generated 283 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:01.167419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:01.232877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf535-60c2-4abe-97b7-a29296f50112 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:01.270987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:01.271773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:01.272054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:01.272324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.275776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:01.311457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:01.349114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.351531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.423794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0318d-92ff-4feb-9f48-a9098e1856e3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:01.484879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 135/264] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:35:01.489373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:01.509200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:01.509822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:01.510104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:01.510380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.566625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:01.695277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0c974-1db0-4c66-a999-cb0be3b93a20 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:01.700561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.702670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.781794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:01.782829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:01.783298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:01.783823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.786468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.788145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.796379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 136/265] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:01 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:01.803612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:01.849686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:01.914778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 130/266] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 747 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:01.918888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:02.004745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88aeb-d760-4982-96b8-028df4834523 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:02.007329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:02.067825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:02.068781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.070272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.070738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.072385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=53152c4c72a940d0ad29a99e40bbf6aa) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:02.073099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:02.076171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:02.090009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 137/267] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:01 2022] GET /identity/v3/role_assignments?user.id=53152c4c72a940d0ad29a99e40bbf6aa&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:02.101296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec658d2-b7de-4150-8927-62e5614c725b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:02.158609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:02.159673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.160146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.160609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.162035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:02.163769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273a80-2294-4026-b06b-55021f25b7f8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:02.171207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 131/268] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:01 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:02.172613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:02.353487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee38c17-0f07-4018-b84e-6ad1b8e3ce5d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:02.429597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:02.430222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.430501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.430773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.431533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=53152c4c72a940d0ad29a99e40bbf6aa) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:02.431851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:02.433295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537a74-e44b-43a9-b512-61efc328c7e3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:02.444394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 138/269] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:02 2022] GET /identity/v3/role_assignments?user.id=53152c4c72a940d0ad29a99e40bbf6aa&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:02.586842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a051d-516a-41d4-870d-07d3214a5326 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.586842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a051d-516a-41d4-870d-07d3214a5326 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.586842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a051d-516a-41d4-870d-07d3214a5326 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.586842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 132/270] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:02.593472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a3a5-96aa-4b75-bf91-fc30983d6027 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.593746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a3a5-96aa-4b75-bf91-fc30983d6027 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.594025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a3a5-96aa-4b75-bf91-fc30983d6027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.616774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6452a3a5-96aa-4b75-bf91-fc30983d6027 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:02.617911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6452a3a5-96aa-4b75-bf91-fc30983d6027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:02.643910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:02.698417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec755f1a-2e3c-4da6-9358-146fe9d04657 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.698697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec755f1a-2e3c-4da6-9358-146fe9d04657 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.698971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec755f1a-2e3c-4da6-9358-146fe9d04657 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.705031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 133/271] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:02.706256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750c8e-b243-40e9-b06c-a2bd245d5548 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.706537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750c8e-b243-40e9-b06c-a2bd245d5548 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.706803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d750c8e-b243-40e9-b06c-a2bd245d5548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.741840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6d750c8e-b243-40e9-b06c-a2bd245d5548 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:02.744210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d750c8e-b243-40e9-b06c-a2bd245d5548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:02.762986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 139/272] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:02.786071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:02.835597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 134/273] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:02.892798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9254c-2f03-46e0-bf4c-985d8dc0f9b7 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.893081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9254c-2f03-46e0-bf4c-985d8dc0f9b7 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.893257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9254c-2f03-46e0-bf4c-985d8dc0f9b7 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.894740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 140/274] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:02.907505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:02.907505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:02.907505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:02.925377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:02.926882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:03.013597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 135/275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:03.129792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9254c-2f03-46e0-bf4c-985d8dc0f9b7 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.139340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.249263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9254c-2f03-46e0-bf4c-985d8dc0f9b7 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.254197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.260782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d140ac-e24f-476b-9a8d-c989f0d742fd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.287881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.288624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.288912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.289187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.290307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.291793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd55601-d346-40b5-9fe5-30413100390e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:03.293522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 141/276] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:03.294477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcb128-23b2-4f9c-8098-b1dbeb474f6f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.294840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcb128-23b2-4f9c-8098-b1dbeb474f6f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.295120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcb128-23b2-4f9c-8098-b1dbeb474f6f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.297655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 142/277] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:03.299137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.299287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcb128-23b2-4f9c-8098-b1dbeb474f6f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.299367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.299367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.299982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.311670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-ab46c27b-0a0c-403b-84e0-f2a304618dd8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:03.316721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:03.317601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:03.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 136/278] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:03.324004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.324283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.324805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.328289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.341797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:03.342990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:03.403469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcb128-23b2-4f9c-8098-b1dbeb474f6f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.409263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 137/279] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:03.410150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.440138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.440747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.441021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.441288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.441929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.443368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:03.470556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 143/280] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:03 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:03.470556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.546871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.570396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c3437-6b5a-483b-a34a-64a23fc13b6a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.575010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.596943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543279-e70b-4f32-bae5-3229fc0bd5fa None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.619125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.619955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.620325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.621045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.636439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:03.638043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.639511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.639799 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.640070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.640732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:03.641032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.642414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:03.642758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:03.643620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:03.644453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 138/281] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:03.649937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.655458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 144/282] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:03.661018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.767424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12f463-ee4a-4565-b1b5-6c8b1f86e6da None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.787704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8227-0ca0-4764-9600-2869fa395c1b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.809194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.810090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.811118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.811632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.812301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:03.812598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.813989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:03.823294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 139/283] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:03.828342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.829707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.830363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.830645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.830921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] PATH_INFO: `/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.832113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] RBAC: Authorizing `identity:get_user(user_id=5ca2e6a88d6b48a0803bf97cf5f5c03f)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.833561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:03.835327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 145/284] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:03.836733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.843426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.862053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:03.951496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb0622-c3b9-422b-a08d-75f70f25612c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.960354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432dc6f-11e7-44f9-9202-8abe40d9f92e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.000421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.001071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.001354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.001628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] PATH_INFO: `/v3/users/2bd8625bebb24b25b9671d6b04914abf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.011448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.012915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.014297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.014766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.018574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] RBAC: Authorizing `identity:get_user(user_id=2bd8625bebb24b25b9671d6b04914abf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.020241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.022099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 146/285] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/users/2bd8625bebb24b25b9671d6b04914abf => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:04.023073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.034025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:04.040871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:04.041803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:04.042610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 140/286] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:04.043284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.111850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.139929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.145029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e4700-1ef3-4427-8ada-af8aa3e67f13 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.180195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dee77-1676-4cc8-b96f-05b3308f0938 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.183234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.184172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.184653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.185127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.187120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.188867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.190842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 147/287] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:04.191902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.224899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.225920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.226423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.226911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.227993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, user.id=2bd8625bebb24b25b9671d6b04914abf, scope.system=all) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:04.228505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.230260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.236390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 141/288] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/role_assignments?user.id=2bd8625bebb24b25b9671d6b04914abf&scope.system=all&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:04.244695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.313144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2e5b8-bb97-4120-bb4f-1eb21d3df719 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.360402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.361290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.361561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.361828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.362593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:04.362995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.364405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.382140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 148/289] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:04.389093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.394424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.409951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec91d8-8acb-432f-bad3-74a6770f7f10 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.443627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.453193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.454050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.454335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.454610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/users/5ca2e6a88d6b48a0803bf97cf5f5c03f/roles/3abef7f219834a45b83613fe1f60cedd` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.456667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, user_id=5ca2e6a88d6b48a0803bf97cf5f5c03f, role_id=3abef7f219834a45b83613fe1f60cedd)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.458149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.474696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 142/290] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:04 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/users/5ca2e6a88d6b48a0803bf97cf5f5c03f/roles/3abef7f219834a45b83613fe1f60cedd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:04.534032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a421c-e3f4-4dc3-96dd-b1084576773c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.572728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.573293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.573567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.573835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] PATH_INFO: `/v3/users/a24bbf548bf440fe9e64df14d7f97ed5` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.575004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] RBAC: Authorizing `identity:get_user(user_id=a24bbf548bf440fe9e64df14d7f97ed5)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.577303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.579009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 149/291] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/users/a24bbf548bf440fe9e64df14d7f97ed5 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:04.585104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:04.709476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3f1b-2030-44f5-8256-06476ee01521 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:04.745790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:04.746398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:04.746670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:04.746935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:04.821075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:04.831955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:04.833858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.965368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 143/292] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:04.971471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:05.044695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:05.075753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb4bd-4c99-44e4-93e6-9e5ea9aa05ca None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:05.111427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:05.112007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.112283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.112550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:05.113662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:05.115112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126abccc-3b76-44c8-9e47-e0298ec00998 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:05.121620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 150/293] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:04 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:05.123589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:05.242331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:05.253643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ddc67d-ab99-4da2-9923-5349baf89084 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:05.289836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:05.290458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.290731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.290997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:05.291739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=a24bbf548bf440fe9e64df14d7f97ed5) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:05.292042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:05.293469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b3abd-025b-4f43-ab58-ff968984febc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:05.306957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 144/294] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:05 2022] GET /identity/v3/role_assignments?user.id=a24bbf548bf440fe9e64df14d7f97ed5&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:05.760394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea045f-e461-4e74-b973-e442dc0e0294 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.761012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea045f-e461-4e74-b973-e442dc0e0294 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.761012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea045f-e461-4e74-b973-e442dc0e0294 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:05.762277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 151/295] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:05.769084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d445b-43cf-41cb-ac91-b9f15eded29e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.769574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d445b-43cf-41cb-ac91-b9f15eded29e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.769651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d445b-43cf-41cb-ac91-b9f15eded29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:05.789902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-503d445b-43cf-41cb-ac91-b9f15eded29e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:05.791073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-503d445b-43cf-41cb-ac91-b9f15eded29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:05.875115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 145/296] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:05.975774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b054-5941-45d4-99eb-4fb8a6ba7295 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.976059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b054-5941-45d4-99eb-4fb8a6ba7295 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.976331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b054-5941-45d4-99eb-4fb8a6ba7295 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:05.978052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 152/297] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:05.984010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:05.984298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:05.984572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.003696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:06.004918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:06.073675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 146/298] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:06.153902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2b054-5941-45d4-99eb-4fb8a6ba7295 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.261271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2b054-5941-45d4-99eb-4fb8a6ba7295 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.300601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.301241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.301516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.301781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.308877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:06.313593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:06.318330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:06.319291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:06.320136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 153/299] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:06.325490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.341539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:06.404820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.436373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caa2db-591e-4c9a-919c-a78c082cf836 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.480287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.480896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.481168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.481436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.482921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:06.483221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:06.484804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:06.499570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 147/300] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:06.502488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.555288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452642e8-e714-4eb4-8936-a641f9e8b271 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.603388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.603973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.604252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.604519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.606889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:06.608208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:06.611214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:06.617506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 154/301] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:06.618351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.640169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544bc8-9250-45f7-b4d2-b66df19da2ff None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.686379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.687015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.687290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.687641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] PATH_INFO: `/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.688832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:06.702201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb515a3fa8884f73b9bbd4b5675dcf2e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:06.703681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea21d5-21e8-4734-81e1-f0e0f5e1e5d3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:06.705496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 148/302] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:06 2022] GET /identity/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:06.707427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772f12-74c3-402c-a20f-51a83a79b8f6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.708427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772f12-74c3-402c-a20f-51a83a79b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.708902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5772f12-74c3-402c-a20f-51a83a79b8f6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.711151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 149/303] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:06.712380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5772f12-74c3-402c-a20f-51a83a79b8f6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:06.735120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1649-2ac8-4995-9a16-4a9d96ff02cd None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.769247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.770515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.770792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.771062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] PATH_INFO: `/v3/groups` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.771717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:06.773144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731564c-987f-4bbe-ab2b-c8e76fba36d9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:06.790497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 155/304] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 24 17:35:06 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:06.790497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329c72-047e-459c-945f-6dd7cdbfe13e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.790497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329c72-047e-459c-945f-6dd7cdbfe13e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.790497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e329c72-047e-459c-945f-6dd7cdbfe13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.816477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8e329c72-047e-459c-945f-6dd7cdbfe13e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:06.817821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8e329c72-047e-459c-945f-6dd7cdbfe13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:06.846639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5772f12-74c3-402c-a20f-51a83a79b8f6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:06.886915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:06.887516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.887817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.888055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.889342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, user.id=cb515a3fa8884f73b9bbd4b5675dcf2e, scope.system=all) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:06.889648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:06.891196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:06.893078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:06.901583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 156/305] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:06.901583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ded4-d14b-4ae3-916e-4eaca0a724eb None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.901583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ded4-d14b-4ae3-916e-4eaca0a724eb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.901583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87ded4-d14b-4ae3-916e-4eaca0a724eb None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.901583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 157/306] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:06.903916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:06.903916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:06.903916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:06.916166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 150/307] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 17:35:06 2022] GET /identity/v3/role_assignments?user.id=cb515a3fa8884f73b9bbd4b5675dcf2e&scope.system=all&role.id=3abef7f219834a45b83613fe1f60cedd => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:06.924724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:06.926090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:06.992061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:07.014388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 158/308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:07.171962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.277643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3fd30-b2f2-4848-8e73-8acfc0885286 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.286256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.314681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.315310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.315614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.315882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.327020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:07.332001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:07.332889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:07.333680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 151/309] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:07.339278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.392480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fe603-5b01-4aa1-b591-99a390f4541c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.403126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:07.429030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.429903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.430327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.430739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.441809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:07.446758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:07.447850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:07.448792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:07.449464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71477f-7245-4849-8ba0-4a7df1426d49 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.451089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 159/310] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:07.459117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.493683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.494359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.494601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.494864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.495539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:07.495920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:07.497217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:07.510767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 152/311] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:07.511925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.529807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:07.587091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc1d1-90f5-41a8-8940-d02163a5c92e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.622303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c48c-42f8-45fd-8d86-fdaf9c392827 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.623897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.624712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.625179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.625698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.626537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:07.627236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:07.628811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:07.642455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 160/312] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:07.643590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.644489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:07.661189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.661900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.662202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.662477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] PATH_INFO: `/v3/users/2bd8625bebb24b25b9671d6b04914abf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.663693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=2bd8625bebb24b25b9671d6b04914abf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:07.665225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:07.667520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 153/313] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/users/2bd8625bebb24b25b9671d6b04914abf => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:07.672584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.758346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697ec4b-0bb9-4754-ad57-c7b1c07806ea None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.768605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:07.783139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7dbaf-b4b4-4d62-976f-492a47fa4cc1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.795138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.795943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.796230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.796636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] PATH_INFO: `/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.797968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca2e6a88d6b48a0803bf97cf5f5c03f)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:07.799582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:07.801330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 161/314] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/users/5ca2e6a88d6b48a0803bf97cf5f5c03f => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:07.806696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:07.822522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.823466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.823985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.824429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] PATH_INFO: `/v3/system/users/2bd8625bebb24b25b9671d6b04914abf/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:07.826610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2bd8625bebb24b25b9671d6b04914abf, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:07.828296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:07.840803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 154/315] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 24 17:35:07 2022] PUT /identity/v3/system/users/2bd8625bebb24b25b9671d6b04914abf/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:35:07.927614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb76aa1-8918-4913-af66-7fe48a3df9a4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:07.967743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:07.968780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:07.969239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:07.969728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.022936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:08.049440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:08.050874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:08.158818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 162/316] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:08.164811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:08.222075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:08.274123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513b0ec-e068-41ec-9717-0a831150b8de None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:08.309498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:08.310135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.311033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.311312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.312483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:08.313936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49708-0bc6-46a5-96c9-5299efcf707c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:08.315612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 155/317] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:08 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:08.321962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:08.402061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:08.427128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a64e60-c95c-45d1-9816-753548f61339 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:08.462196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:08.462804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.463183 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.463446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.464202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, scope.project.id=a270bf9f70d148479b3cf67258e66222, user.id=5ca2e6a88d6b48a0803bf97cf5f5c03f) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:08.464512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:08.466024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c1ccc-9901-4da3-9471-5b0e0b670129 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:08.472334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 163/318] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:08 2022] GET /identity/v3/role_assignments?user.id=5ca2e6a88d6b48a0803bf97cf5f5c03f&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=3abef7f219834a45b83613fe1f60cedd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:08.835789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d3e3-d04b-4cf8-af05-006aeb580573 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.836130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d3e3-d04b-4cf8-af05-006aeb580573 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.836326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d3e3-d04b-4cf8-af05-006aeb580573 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.837771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 156/319] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:08.843202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.843672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.843755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.861434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:08.862599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:08.903984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11dab-b20a-4bdd-945e-3e0bb3a11a6c None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.904859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11dab-b20a-4bdd-945e-3e0bb3a11a6c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.905371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f11dab-b20a-4bdd-945e-3e0bb3a11a6c None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.906580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 157/320] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:08.911350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.911668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.911946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.930373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:08.931220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 164/321] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:08.931630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:09.002807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 158/322] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:09.200886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.268772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.281029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.303263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1853c73-c953-47d3-8af0-9f2b463c6b5a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.333752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.339915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.340556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.340836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.341291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.352009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:09.356747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:09.358369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:09.359194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 165/323] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:09.364343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.394680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ec598-9c64-433c-a110-45b7e45576ea None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.430682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.431326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.431637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.431904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.433008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.434462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.439991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 159/324] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:09.442092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.473890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a751-9fa6-49c6-9133-4923c18b6298 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.505652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.512555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.513178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.513489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.513757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.514482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:09.514796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.518482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.526835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 166/325] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:09.532387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.561059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dfa99-7e01-415c-b6ae-23c52a7bc38c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.584941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.608515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.609140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.609419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.609691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] PATH_INFO: `/v3/groups` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.610312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.612065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.631513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 160/326] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Aug 24 17:35:09 2022] POST /identity/v3/groups => generated 226 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:35:09.643076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab09a1-be82-4a9a-8316-d5f70d692a10 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.681308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.681972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.682250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.682524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] PATH_INFO: `/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.684093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] RBAC: Authorizing `identity:get_user(user_id=cb515a3fa8884f73b9bbd4b5675dcf2e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.685735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d910-45c4-43df-91b9-b593827e9e13 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.687625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 167/327] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:09 2022] GET /identity/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:09.693448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.804205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b2142-e897-4128-b42c-15b6bbf383b7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.838503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.839162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.839580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.839798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] PATH_INFO: `/v3/system/users/cb515a3fa8884f73b9bbd4b5675dcf2e/roles/3abef7f219834a45b83613fe1f60cedd` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.841175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cb515a3fa8884f73b9bbd4b5675dcf2e, role_id=3abef7f219834a45b83613fe1f60cedd)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.843187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c751827-c163-46de-9f65-e405c1609ad5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.850417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 161/328] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Aug 24 17:35:09 2022] PUT /identity/v3/system/users/cb515a3fa8884f73b9bbd4b5675dcf2e/roles/3abef7f219834a45b83613fe1f60cedd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:09.852454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88a03-25e1-48d1-ae04-661708a12161 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.852892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88a03-25e1-48d1-ae04-661708a12161 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.853056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c88a03-25e1-48d1-ae04-661708a12161 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.854400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 168/329] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:09.861776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.861972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.862198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.882637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:09.883991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:09.925824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.965752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 162/330] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:10.240815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88a03-25e1-48d1-ae04-661708a12161 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.309271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:10.364708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c88a03-25e1-48d1-ae04-661708a12161 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.409141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.409792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.410074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.410351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.425529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:10.430763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:10.431939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:10.432732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 169/331] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:10.437725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.587991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70b051-b437-4ad3-989f-c1ddce9f3b96 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.631305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.632291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.632768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.633236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.634088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:10.634769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.636385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.647380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 163/332] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:10.654133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.797121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0089e-4fff-4489-b6fd-81d0a07f15c2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.827402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:10.843680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.844329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.844610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.844882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] PATH_INFO: `/v3/users/2bd8625bebb24b25b9671d6b04914abf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.846027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] RBAC: Authorizing `identity:get_user(user_id=2bd8625bebb24b25b9671d6b04914abf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.847496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17e79d-ccb6-41d6-8eac-8523475d11a0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.849393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 170/333] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/users/2bd8625bebb24b25b9671d6b04914abf => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:10.855342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.911836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:10.994569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ba40-abbc-4623-92d3-72caf453d1dd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:11.041139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:11.041773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:11.042052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:11.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:11.043057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] RBAC: Adding query filter params (role.id=7e7dcf0e0b644abb9ae0549320fd777e, user.id=2bd8625bebb24b25b9671d6b04914abf, scope.system=all) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:11.043363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:11.044921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fb734-6063-403d-98e0-06a72ad90544 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:11.063618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 164/334] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/role_assignments?user.id=2bd8625bebb24b25b9671d6b04914abf&scope.system=all&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:11.976179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3818-032a-4b90-8bd6-c8fa2152aa4e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:11.976729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3818-032a-4b90-8bd6-c8fa2152aa4e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:11.976729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3818-032a-4b90-8bd6-c8fa2152aa4e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:11.978479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 171/335] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:11.988708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1756e-3e44-4419-97ce-3e02b152b369 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:11.989168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1756e-3e44-4419-97ce-3e02b152b369 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:11.989337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1756e-3e44-4419-97ce-3e02b152b369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.012610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a2c1756e-3e44-4419-97ce-3e02b152b369 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:12.014378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a2c1756e-3e44-4419-97ce-3e02b152b369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:12.099681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 165/336] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:12.179279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dea89-f155-4828-be6f-92d6f44bfb62 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:12.179627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dea89-f155-4828-be6f-92d6f44bfb62 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:12.179832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dea89-f155-4828-be6f-92d6f44bfb62 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.181181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 172/337] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:12.187454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:12.187828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:12.188102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.207164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:12.208439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:12.231142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:12.292171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 166/338] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:12.498789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dea89-f155-4828-be6f-92d6f44bfb62 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:12.635444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dea89-f155-4828-be6f-92d6f44bfb62 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:12.675244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:12.675992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:12.676273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:12.676543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.696468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:12.701800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:12.702663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:12.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 173/339] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:12.708779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:12.782531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:12.835297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:12.853295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f26ad3-4018-4ff2-86e4-ce708b5f3dc7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:12.898397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fc719-b62d-4129-9ff3-326c47da866d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:12.914717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:12.915626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:12.916086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:12.916525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.917390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:12.917887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:12.919565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:12.934446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 167/340] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:12.941426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:12.941821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:12.942068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:12.942507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:12.942922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:12.957821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:12.960711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:12.966226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:12.967827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:35:12.968610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:12.970855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 174/341] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:12.976919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:13.083462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dfdc8-b31d-4cc2-93a7-5f88b814aa3e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.104125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b1c3a-011d-4150-bff2-cc5195c3461f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.114763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.125887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.126806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.127231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.127742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.142308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.143134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.143640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.144047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.144892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:13.145346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.146863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c2b5-add7-4e1a-b0ac-28a6c23fb722 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.157927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 175/342] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:13.159356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae913f-3882-4792-83eb-98a9c940a02d None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.163832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae913f-3882-4792-83eb-98a9c940a02d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.164493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae913f-3882-4792-83eb-98a9c940a02d None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.166918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 176/343] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:13.168922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae913f-3882-4792-83eb-98a9c940a02d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:13.206215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.207813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.223739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.282954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae913f-3882-4792-83eb-98a9c940a02d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.319724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.320728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.321204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.321673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] PATH_INFO: `/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.323124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] RBAC: Authorizing `identity:get_user(user_id=cb515a3fa8884f73b9bbd4b5675dcf2e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.324790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a773318-cf57-4d22-9db6-134e4f9c2b76 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.331668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 177/344] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:35:13 2022] GET /identity/v3/users/cb515a3fa8884f73b9bbd4b5675dcf2e => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:13.333209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.333746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.334268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.353728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 168/345] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:35:13.353728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:13.362272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:13.364122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:13.396844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.458822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 178/346] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:13.460345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:13.510280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c1cf-5496-49cd-8ae2-bb91d2546ce3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.530267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.553039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.553270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.553634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.553975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.554829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] RBAC: Adding query filter params (role.id=3abef7f219834a45b83613fe1f60cedd, user.id=cb515a3fa8884f73b9bbd4b5675dcf2e, scope.system=all) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:13.555136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.556727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c823c25-c462-4756-8cde-8c95247e00f5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.570476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 169/347] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Aug 24 17:35:13 2022] GET /identity/v3/role_assignments?user.id=cb515a3fa8884f73b9bbd4b5675dcf2e&scope.system=all&role.id=3abef7f219834a45b83613fe1f60cedd => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:13.575526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26f01-8edd-4e5e-9152-d875385520ac None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.575526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26f01-8edd-4e5e-9152-d875385520ac None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.575526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b26f01-8edd-4e5e-9152-d875385520ac None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.575526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 170/348] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:13.580060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.580345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.580635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.600513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:13.602109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:13.605100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dae28a-9bfe-4aed-8564-2b76c25a1ac3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.657213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.658215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.658712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.659178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.660581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.662214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.664239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 179/349] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:13 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:13.670459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:13.681191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 171/350] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:13.722906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:13.814568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e78bc-f461-45dc-89c6-89d8d797e4fe None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:13.856858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:13.858480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:13.859059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:13.859547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:13.869330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:13.871020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:13.873481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 180/351] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:13 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:13.880053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.025651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.046933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d01dd-6c8a-4454-adcd-6adb42104282 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.052697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.103031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.103704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.104020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.104181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.104895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=139d28049ff14e92b984e2e272c11ae8) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:14.105205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.106675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.113277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 172/352] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:13 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:14.114043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.171857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.176818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5264a-bf3e-4dfa-b683-d3991fd05e6e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.246842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.249340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.250041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.250991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.259052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94de62-1698-4ef0-94b5-86995c1ab707 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.270382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-94864996-0d35-427f-813b-9ae125fdc551 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:14.279780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:14.280771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:14.281590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:14.284912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 181/353] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:14.287194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cb226-b9d4-41f6-ac7d-9b4b00b9ed96 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.287723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cb226-b9d4-41f6-ac7d-9b4b00b9ed96 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.288514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cb226-b9d4-41f6-ac7d-9b4b00b9ed96 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.290477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 182/354] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:14.292063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cb226-b9d4-41f6-ac7d-9b4b00b9ed96 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.310301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.311550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.311835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.312109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.328997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b12a7ec9-0278-4ba3-9ed6-a3fea6a4dbb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:14.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:14.335053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:14.335881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:14.336482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 173/355] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:14.337459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.337740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.338011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.369082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:14.371928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:14.403534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.426028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cb226-b9d4-41f6-ac7d-9b4b00b9ed96 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.465788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.466452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.466920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.467191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.467902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:14.468203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.469832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.475055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 174/356] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:14.476190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.481011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 183/357] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:14.485578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.515757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.609462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644ffc7-2db0-4949-919b-f1eb9fa8e7ee None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.654382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032ae27-a273-43ed-a227-813e2774d828 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.657184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.657804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.658080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.658350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.659017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:14.659315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.661752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.681139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 175/358] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:14.687588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.715598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.720512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.721356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.721784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.722198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.810086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.812231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc549d-a358-4952-a02c-0ee9870b66b0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.821654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653df12-c0ab-46aa-bdc1-b02078f27711 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:14.844041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:14.870246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:14.871274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.871715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.871989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.938886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 184/359] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:14.940918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9f2a-c1df-44ab-8fc2-3b23e76cfd52 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:14.941208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9f2a-c1df-44ab-8fc2-3b23e76cfd52 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:14.941479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc9f2a-c1df-44ab-8fc2-3b23e76cfd52 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:14.942857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 185/360] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:14.943428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc9f2a-c1df-44ab-8fc2-3b23e76cfd52 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:14.965083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:14.967618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf8efb-54fd-464e-903f-9ca01a81b8af None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:14.995400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.073527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc9f2a-c1df-44ab-8fc2-3b23e76cfd52 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.103367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.103679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 176/361] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:35:15.104826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.105112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.105384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.124190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.124834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.125109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.125383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.129156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:15.130384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:15.137227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:15.143393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:15.144746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:15.145565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 186/362] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:15.146248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.210472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 177/363] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:15.211237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.267768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.281272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb462437-9f1f-46bb-a308-4a42cb7f1b64 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.324133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6001c-d369-4535-a3c2-2ce4aba02a65 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.327543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.328180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.328453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.328716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.329831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.331270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.337649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 187/364] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:15.338509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.363887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.364534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.364810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.365079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.366410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.368062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.370244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 178/365] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:15.371047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.394078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.448673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf92b-fca5-4821-a13d-6b7df8650f91 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.485851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.486714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.486984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.487251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.487932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:15.488227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.489593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.495635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b36109-9f97-4a8a-84c2-acc1a44f5b6f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.498330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 188/366] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:15.500104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.519830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.554434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.555171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.555453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.555804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.556938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.558372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.559992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 179/367] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:15.560996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.631948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5fb50-83d8-4f84-8b5a-39cf4c476895 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.647697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.680378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.681020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.681300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.681467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a317b-b497-48a0-a396-543127cb4c1c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.681623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.682699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.684205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.685906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 189/368] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:15.686747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.730521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.731135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.731412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.733899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.797540 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:15.819319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:15.820000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afbbfb-aa51-40f5-a005-2b15a0e04ee4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:15.820888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:15.872221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:15.873863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:15.874350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:15.874929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:15.890199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:15.896600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:15.897376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 190/369] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:15.898646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.955967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 180/370] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:15.957167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:15.964270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.043597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32748-30fc-4579-997e-76d5e90b7842 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.069652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.086306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.087026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.087377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.087783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.088585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=139d28049ff14e92b984e2e272c11ae8) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:16.088964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.090970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.094219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac9562-87e9-4a5d-91e4-717f5c885acf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.100314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 191/371] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.101047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.151649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.152260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.152534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.152799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.153526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=a270bf9f70d148479b3cf67258e66222) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:16.153955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.156069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.162977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 181/372] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:15 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.168880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.252001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f35b70c-f1d8-40c3-a85d-a71f020e5686 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.255765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.299816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.301152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.301448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.301716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.302423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:16.302730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.304512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.317275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 192/373] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.324278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.352038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d10a-0b05-40fb-a388-239686aa7936 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.442387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.443033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.443311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.443635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.445960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.447404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669fdef-53d5-432d-a23c-d81761f1bdeb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.450231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 182/374] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.451442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acc49f-60f4-414e-9a5c-15fce1db94e7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.451746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acc49f-60f4-414e-9a5c-15fce1db94e7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.452019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acc49f-60f4-414e-9a5c-15fce1db94e7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.453377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 183/375] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:16.461139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acc49f-60f4-414e-9a5c-15fce1db94e7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.476387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7742b-aa8b-4dc9-94af-b9e8c7cf7452 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.490639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.525223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.525995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.526379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.526721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.572491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.613592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acc49f-60f4-414e-9a5c-15fce1db94e7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.617719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.618814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.660847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.661826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.662285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.662735 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.664199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.666411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18545b-1758-4cdb-abb8-629f46113d28 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.669014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 184/376] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.671083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.671583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.675564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.704716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:16.706458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:16.752046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 193/377] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:16.752943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.799184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 185/378] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:16.804179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:16.828591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.842667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:16.875972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba383ad-4aa1-4a59-a67b-5073f3eb494f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.930536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.931266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.931567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.932005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.932964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=a270bf9f70d148479b3cf67258e66222) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:16.933392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.933702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99726a2-bf9a-417d-b3f1-259e2ac734d5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:16.934880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.946344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 194/379] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.972176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:16.972827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:16.973109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:16.973378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] PATH_INFO: `/v3/projects/c0a52b9c017649d7b6608817068a47f8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:16.974556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] RBAC: Authorizing `identity:get_project(project_id=c0a52b9c017649d7b6608817068a47f8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:16.976055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:16.977910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 186/380] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/projects/c0a52b9c017649d7b6608817068a47f8 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:16.983993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.118086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f6ed3-5f05-428f-986b-0af12145cd40 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.156013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:17.161842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.162485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.162762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.163031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] PATH_INFO: `/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.172622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] RBAC: Authorizing `identity:get_group(group_id=9ccb885cc1e64536b5b9c4a5470f9626)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.174081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:17.176086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 195/381] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:16 2022] GET /identity/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:17.181839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.235746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.297596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:17.334348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8f3f8-f66f-4112-95cf-5346b4640957 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.412342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.412929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.413211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.413481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.414205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] RBAC: Adding query filter params (group.id=9ccb885cc1e64536b5b9c4a5470f9626, role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=c0a52b9c017649d7b6608817068a47f8) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:17.414508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.416108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:17.423973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 187/382] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:17 2022] GET /identity/v3/role_assignments?group.id=9ccb885cc1e64536b5b9c4a5470f9626&scope.project.id=c0a52b9c017649d7b6608817068a47f8&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:17.464875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde37fc-c93e-44df-af68-2fa42a10fb56 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.548276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.548947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.549222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.549489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.565584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:17.570617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:17.571586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:17.572430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:17.577063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 196/383] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:17.578528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.722358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251b37-feeb-476a-9018-cfc6d5232107 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.759360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.760368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.760862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.761337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.762489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] RBAC: Adding query filter params (name=member) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:17.763008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.764658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:17.777640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 188/384] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:17.783256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.882883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:17.937381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42c397-46a5-478a-b6ca-4c2853137338 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.981309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.981876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.982684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.982955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.988144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.989606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:17.991546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 197/385] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:17 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:17.996857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.016887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:18.165244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421546-2a21-4c6c-b68d-1b1d0d9ea12a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.209078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.210528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.211011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.211530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.212861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.217988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea54d4-33df-4604-9fa1-01d6a2f0eb82 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.220304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 189/386] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:17 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:18.226244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.366863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5090c-ea73-412c-9060-49b716f8231c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.394067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:18.406149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.406815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.407093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.407366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.412808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.414282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a20030-3333-428d-b386-fa51236f5506 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.447802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 198/387] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 17:35:18 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:18.641132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a3fb2-c21d-4d0b-9ab9-ed80b6569da0 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.641310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a3fb2-c21d-4d0b-9ab9-ed80b6569da0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.641584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a3fb2-c21d-4d0b-9ab9-ed80b6569da0 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.643348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 190/388] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:18.648889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.649178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.649455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.670775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:18.672017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:18.716265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fbd22-4073-42cc-8510-b9617adff360 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.717115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fbd22-4073-42cc-8510-b9617adff360 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.717455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65fbd22-4073-42cc-8510-b9617adff360 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.719080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 191/389] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:18.735334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.735644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.736392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.765582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 199/390] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:18.768991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:18.770192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:18.842876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 192/391] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:19.185712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.199038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.213726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.302936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467b1e7-7c49-4f90-9568-9db5437ad653 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.318985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.356972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.357642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.357917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.358186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.371290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:19.384717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:19.385673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:19.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:19.389172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a566ae9-6713-4940-b49d-b33607095356 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.390083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 200/392] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:19.395438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.433701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.434353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.434630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.434897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.458127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:19.462898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:19.463835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:19.464623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 193/393] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:19.472643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.595582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.607940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e853f6-8994-442d-8612-7f8108b3790c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.642257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c894a1f-3d95-41a3-b1cb-013123ef77c9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.653537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.654803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.655266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.655673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.656429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:19.656824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:19.658427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf802-76a4-49fc-9321-f15ef2eed2f5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:19.667374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 201/394] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:19.668756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230d1c-ece2-4ab6-83a5-2bd9f505340c None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.669135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230d1c-ece2-4ab6-83a5-2bd9f505340c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.669620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0230d1c-ece2-4ab6-83a5-2bd9f505340c None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.675639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 202/395] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:19.676550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230d1c-ece2-4ab6-83a5-2bd9f505340c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.682845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.683457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.683769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.684038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.684691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:19.684989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:19.686480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c5d98-7f2d-46ba-8526-be4f9f1f0c61 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:19.699702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 194/396] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:19.701099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.701381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.701656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.715199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.723369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:19.724713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:19.788203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0230d1c-ece2-4ab6-83a5-2bd9f505340c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.798869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 195/397] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:19.800080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.845152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.845801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.846165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.846447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.847585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:19.849102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:19.851079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 203/398] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:19.856233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.862743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:19.909483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e6b9a-e266-4f7e-a78e-5c0b3c540431 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.946409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:19.947302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.947774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.948245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.949513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:19.951864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d1744-e646-49f6-9020-53d1feeecac7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:19.953648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 196/399] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:19.955060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc6e73-c1d1-4f83-8c0d-202b83c1b7d0 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:19.955495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc6e73-c1d1-4f83-8c0d-202b83c1b7d0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:19.955956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc6e73-c1d1-4f83-8c0d-202b83c1b7d0 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:19.957431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 197/400] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:19.964367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc6e73-c1d1-4f83-8c0d-202b83c1b7d0 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:19.976646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32078-8a45-4092-aa79-9d0df51887a0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:19.978181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.013579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.014217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.014497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.014765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.015965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.017430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b11cbf-f405-4c23-a847-4d748dfea3cf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.019383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 204/401] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:20.020655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.020931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.021545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.039743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:20.041105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:20.080647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc6e73-c1d1-4f83-8c0d-202b83c1b7d0 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.110722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.120715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.121420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.121617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.121887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.123689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.125190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.127997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 205/403] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:20.135190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 198/403] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:19 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:20.135190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.137917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.250978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.254451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed402-6aa8-4201-be7c-55b2db6ec52c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.272519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbce539-6985-4d2d-a7d0-82a8d31dddfb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.291398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.292227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.292685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.293062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.295218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.297040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.313618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.314375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.314609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.314885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.319924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 206/404] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 17:35:20 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:20.321730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.323497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] RBAC: Authorizing `identity:create_grant(project_id=139d28049ff14e92b984e2e272c11ae8, group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.326466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.353885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 199/405] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 17:35:20 2022] PUT /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:20.409159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.502654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851426-9625-41f4-94da-1c693848fca7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.522991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.565625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.566282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.566558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.566828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.581817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:20.587202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:20.588401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:20.589355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:20.590134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edbdee-72ab-4ea2-a738-ad1eb7e78d99 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.592802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 207/406] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:20.599170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.631617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.635888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.637779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.638060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.638485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.652596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:20.659137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:20.660086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:20.661515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 200/407] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:20.668212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.800484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2984-bbeb-48f0-837a-5e5a1876f229 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.859303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a425e-543c-4d5b-99ee-8a3a04cb30cb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:20.872434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.873599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.873871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.874136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.874803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:20.875101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.876497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.892656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 208/408] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:20.898487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:20.903571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:20.964847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:20.965480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:20.965762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:20.966039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:20.966799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:20.967104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:20.968523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:20.998311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 201/409] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:21.001348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:21.007570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85986e0c-030f-4fda-8aaf-62027e29ad6a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.008561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.045835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.046636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.046920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.048016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.049161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.050665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3a7e3-1eaa-420f-98a2-8095eccbeffc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.052946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 209/410] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:20 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:21.054523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d44-71b7-4734-a8b7-cb61dd8cb1d9 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.054808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d44-71b7-4734-a8b7-cb61dd8cb1d9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.055086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d44-71b7-4734-a8b7-cb61dd8cb1d9 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.056994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 210/411] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:21.060867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984d44-71b7-4734-a8b7-cb61dd8cb1d9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.134780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5b4f-f8e0-4699-9c44-006335bfb091 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.164963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984d44-71b7-4734-a8b7-cb61dd8cb1d9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.181537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.182344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.182912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.183197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] PATH_INFO: `/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.184424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] RBAC: Authorizing `identity:get_group(group_id=9ccb885cc1e64536b5b9c4a5470f9626)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.186845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f2f99-80e3-4b01-9053-901ebdae613d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.189626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 202/412] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:21 2022] GET /identity/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:21.191397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.191720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.192324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.199572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.200205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.200480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.200779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.202440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.204008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.210470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 211/413] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:21 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:21.211314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.212927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:21.214466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:21.235831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:21.268271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:21.296183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 203/414] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:21.296982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.344130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410e311-7e0a-475a-84e7-6b61405132bc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.406413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.407057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.409945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.411933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] PATH_INFO: `/v3/projects/c0a52b9c017649d7b6608817068a47f8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.413200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] RBAC: Authorizing `identity:get_project(project_id=c0a52b9c017649d7b6608817068a47f8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.414640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.420826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 212/415] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:21 2022] GET /identity/v3/projects/c0a52b9c017649d7b6608817068a47f8 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:21.427279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.440470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb24c54-3825-4d67-8ad5-b40cf9920675 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.556214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.556894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.557165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.557432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.571912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:21.572253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] RBAC: Authorizing `identity:create_grant(project_id=a270bf9f70d148479b3cf67258e66222, group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.573733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.584679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0b9a-9c29-4f8d-8930-d0999e5e3f01 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.602979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 204/416] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 17:35:21 2022] PUT /identity/v3/projects/a270bf9f70d148479b3cf67258e66222/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:21.626446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.627440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.627932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.628300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] PATH_INFO: `/v3/projects/c0a52b9c017649d7b6608817068a47f8/groups/9ccb885cc1e64536b5b9c4a5470f9626/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.630254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0a52b9c017649d7b6608817068a47f8, group_id=9ccb885cc1e64536b5b9c4a5470f9626, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:21.631941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:21.632051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:21.666702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 213/417] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Aug 24 17:35:21 2022] PUT /identity/v3/projects/c0a52b9c017649d7b6608817068a47f8/groups/9ccb885cc1e64536b5b9c4a5470f9626/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:21.692755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:21.792906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2034724-c0ae-4153-b850-161499c9b326 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:21.851314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:21.852054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:21.852334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:21.852658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:21.869685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:21.875147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:21.876049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:21.876896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 205/418] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:21.882414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.002983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30ad1e-a65c-40d6-a31e-dc9d777c5466 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.052712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:22.053729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.054212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.054682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.055571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:22.056093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:22.057694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:22.070661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 214/419] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:22.076525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.193273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.200511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fd80f-ef70-44bd-804b-42620655c81f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.246185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:22.246806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.247088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.247363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.338432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:22.339955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23a566-ed30-4180-8d44-a1713fedd96b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:22.458511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.467126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 206/420] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:22.472268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.491887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.579124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac60a-9b5b-468d-8446-7a8c6f5d43bf None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.592391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bf9-4a17-43ad-aa50-d4eaa7781631 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.592676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bf9-4a17-43ad-aa50-d4eaa7781631 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.592950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89bf9-4a17-43ad-aa50-d4eaa7781631 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.594596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 207/421] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:22.600679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.601081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.601452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.619195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:22.620101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.620543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.620964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.621790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:22.622402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:22.623879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:22.624724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:22.626821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 215/422] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:22 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:22.632874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.718606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:22.755968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 208/423] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:22.757861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4dfc4-9903-43a2-9027-b5104468b12c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.792540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:22.793224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.793510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.793782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.795606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:22.797056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a1d42-cd7c-4d91-9801-12e3bb687c38 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:22.799070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 216/424] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:22 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:22.804861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:22.808916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d163ce4-d1ef-4a11-88bb-ea21ec462305 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.809203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d163ce4-d1ef-4a11-88bb-ea21ec462305 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.809475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d163ce4-d1ef-4a11-88bb-ea21ec462305 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.811040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 217/425] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:22.819504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:22.819806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:22.820081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:22.852499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:22.853717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:22.962676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbb228-d1b0-41da-a18c-c3bc4c57f70d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:22.975710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 218/426] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:23.001380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.002053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.002341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.002619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.003348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=139d28049ff14e92b984e2e272c11ae8) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:23.003689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:23.005198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:23.011607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 209/427] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:22 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:23.012050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.131130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.213750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.241120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7ba60-91c0-4081-b450-cb756dad6153 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.278753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.280018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.280301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.280567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.291974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:23.296760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:23.297621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:23.298419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 219/428] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:23.305188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.330487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.431336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b8078-49c2-441f-b3ab-73af0e73256c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.471855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbe86c-e82b-40a8-8b90-df7e6804b518 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.473572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.474247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.474530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.474801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.475464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:23.475811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:23.477248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:23.487736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 210/429] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:23.493896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.517464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.518129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.518386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.518671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.521978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.536080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:23.542437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:23.543353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:35:23.544192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:23.547099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 220/430] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:23.553932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.573273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.642041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c496-2881-434a-8e0e-ae707042a7a3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.685715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a03b-f0d0-45e4-a6d1-5e7ede3e0182 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.687635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.688521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.690756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.690756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.728514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.729602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.730107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.730664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.731535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] RBAC: Adding query filter params (name=member) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:23.732734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:23.734407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:23.745472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 221/431] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:23.750638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:23.795823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:23.797292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:23.826944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.855514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:23.883810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b26749-0817-4473-94b9-0816fc243b64 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:23.925378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:23.925378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:23.925378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:23.925378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:23.926154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 211/432] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:23.931359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.012354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.013890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d06c20-ecbb-4f58-bfa8-ee76c11187ac None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.054359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75f646-ec99-4158-8e12-85c773e5a3bb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.080264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.095332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.096090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.096362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.096632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] PATH_INFO: `/v3/projects/139d28049ff14e92b984e2e272c11ae8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.097756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] RBAC: Authorizing `identity:get_project(project_id=139d28049ff14e92b984e2e272c11ae8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.099209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16800e-0d64-4b28-a0af-812d804704ff None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.103801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 212/433] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/projects/139d28049ff14e92b984e2e272c11ae8 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:24.105751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94601d7-1eea-48ae-b05c-cb2c9e003a6d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.106044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94601d7-1eea-48ae-b05c-cb2c9e003a6d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.106321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94601d7-1eea-48ae-b05c-cb2c9e003a6d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.108712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 213/434] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:24.108945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94601d7-1eea-48ae-b05c-cb2c9e003a6d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.128556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 222/435] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:24.131109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.132823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.133127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.133665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.150818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:24.152030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:24.224065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94601d7-1eea-48ae-b05c-cb2c9e003a6d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.227846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 223/436] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:24.229011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.261686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.262491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.262709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.262981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.264931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.266431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.268313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 214/437] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:24.274206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.336458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.352338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb30930-d86a-4303-8524-f1638a806abf None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.367632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.384464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4f3d-01d5-4684-889c-63498b873adb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.389220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.389846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.390126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.390399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.391542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.392986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6477f97-671c-4f62-836d-4f0331db5454 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.394614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 224/438] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:24.400156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851347f-36ac-46ea-9dd7-3330026d02af None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.400438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851347f-36ac-46ea-9dd7-3330026d02af None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.400715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851347f-36ac-46ea-9dd7-3330026d02af None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.402064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 225/439] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:24.402661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851347f-36ac-46ea-9dd7-3330026d02af None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.424561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.425169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.425450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.425713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.426426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=139d28049ff14e92b984e2e272c11ae8) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:24.426751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.428193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a17ef2-dbca-4af3-a383-409eb9f24108 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.433866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 215/440] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=139d28049ff14e92b984e2e272c11ae8&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:24.441351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.441780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.443223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.466117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:24.468401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:24.527820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851347f-36ac-46ea-9dd7-3330026d02af None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.549695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 216/441] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:24.550580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.579324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.580291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.580573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.580845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.582074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.584315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.590349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 226/442] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:24.592775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.623442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.667934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.701957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c426535-ad1e-4483-8662-4b327d0c28be None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.719437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f430bbb-4d18-456d-a012-e2f287f9e665 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.752688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.753710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.753997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.754270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.764573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.765178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.765455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.765738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.766205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:24.766434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=7e7dcf0e0b644abb9ae0549320fd777e, scope.project.id=a270bf9f70d148479b3cf67258e66222) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:24.766737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.768205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:24.771235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:24.772187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:24.773036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:24.773647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 217/443] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:24.779394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 227/444] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=7e7dcf0e0b644abb9ae0549320fd777e => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:24.780365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.829955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.906285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd8170-28b4-4128-be9f-cb349d31b29e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.950241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.953534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:24.954569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.954569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.954866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.955951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:24.956327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:24.958088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:24.968532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 218/445] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:24.972454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc854b-13d6-4386-92a3-ef7441f31711 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:24.974037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.976975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.015683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.017135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.017412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.017680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.032291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:25.036925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:25.037828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:25.038632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 228/446] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:25.043812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.099814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86483-74a1-4548-8932-e9e9b077cd30 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.148011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.148632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.148911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.149182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.159894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07111cba-5f98-49d7-aeba-ae7ad78cf9f0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.193433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.194031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.194307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.194577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.195221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:25.195539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.196897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.205423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 229/447] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.210993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.217659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.219103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.220982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.221503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.313282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8fed5-c141-4830-bcce-6609af8ed01d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.321930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 219/448] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:35:25.327794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.347931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.348539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.348816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.349084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.416279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.417738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.424825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9b150-05a3-4b42-9a06-7eef15eab3fd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.426366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.428019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.462305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.462938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.463219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.463505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] PATH_INFO: `/v3/projects/a270bf9f70d148479b3cf67258e66222` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.464660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] RBAC: Authorizing `identity:get_project(project_id=a270bf9f70d148479b3cf67258e66222)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.466111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.468350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 220/449] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/projects/a270bf9f70d148479b3cf67258e66222 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.473876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.522432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 230/450] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:35:25.528281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.579375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214ba4-9608-4e38-b227-f22b90eab19e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.614116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.614751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.615028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.615301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] PATH_INFO: `/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.616452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] RBAC: Authorizing `identity:get_group(group_id=4586b0c7ff874c1c9f8a6a82ba1afd8a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.617907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.619516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 221/451] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/groups/4586b0c7ff874c1c9f8a6a82ba1afd8a => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.625859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.628644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26e759-1be4-4d2d-8e58-8a6f53b7dbc8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.639199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.663609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.664228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.664501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.664769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] PATH_INFO: `/v3/projects/c0a52b9c017649d7b6608817068a47f8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.665292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.665935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] RBAC: Authorizing `identity:get_project(project_id=c0a52b9c017649d7b6608817068a47f8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.667379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.669015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 231/452] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/projects/c0a52b9c017649d7b6608817068a47f8 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.674469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.731944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01893077-a9e6-4657-9ead-c83343f07e23 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.768517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.769116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.769393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.769661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.770386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] RBAC: Adding query filter params (group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a, role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=a270bf9f70d148479b3cf67258e66222) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:25.770688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.773177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.776721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392da7c1-6a42-4c4a-84d5-1cd352636776 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.777887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 222/453] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/role_assignments?group.id=4586b0c7ff874c1c9f8a6a82ba1afd8a&scope.project.id=a270bf9f70d148479b3cf67258e66222&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.816295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.816935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.817214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.817484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] PATH_INFO: `/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.818637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] RBAC: Authorizing `identity:get_group(group_id=9ccb885cc1e64536b5b9c4a5470f9626)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.820138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611776fd-918a-47c5-b1ef-8e42a6a71f5f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.821797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 232/454] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/groups/9ccb885cc1e64536b5b9c4a5470f9626 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:25.829523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:25.936369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d5598-83d6-4024-915f-0e2164209c46 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.937928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.973780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.974379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.974652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.974915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.975708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] RBAC: Adding query filter params (group.id=9ccb885cc1e64536b5b9c4a5470f9626, role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=c0a52b9c017649d7b6608817068a47f8) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:25.976031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.977457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d2e2a-3575-4221-8286-6782a2827c31 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.983800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 223/455] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Aug 24 17:35:25 2022] GET /identity/v3/role_assignments?group.id=9ccb885cc1e64536b5b9c4a5470f9626&scope.project.id=c0a52b9c017649d7b6608817068a47f8&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:28.227947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50475-3d9b-423e-bca6-e82b5383156b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.228497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50475-3d9b-423e-bca6-e82b5383156b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.228576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50475-3d9b-423e-bca6-e82b5383156b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.230160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 233/456] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:28.236198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a33d-38ab-4c4c-913d-374b7110960f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.236591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a33d-38ab-4c4c-913d-374b7110960f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.236796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47a33d-38ab-4c4c-913d-374b7110960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.257628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5e47a33d-38ab-4c4c-913d-374b7110960f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:28.258815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5e47a33d-38ab-4c4c-913d-374b7110960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.361777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 224/457] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:28.584382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f46f-06b5-4802-933c-c51225264713 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.584658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f46f-06b5-4802-933c-c51225264713 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.585178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f46f-06b5-4802-933c-c51225264713 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.586941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 234/458] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:28.593177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c269c-2137-4aa3-bde2-ddb0898cdedf None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.593733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c269c-2137-4aa3-bde2-ddb0898cdedf None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.594205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c269c-2137-4aa3-bde2-ddb0898cdedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.621143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c82c269c-2137-4aa3-bde2-ddb0898cdedf None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:28.623178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c82c269c-2137-4aa3-bde2-ddb0898cdedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.719375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30cf3-8dbd-45d5-a24e-1b3266c55936 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.720046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30cf3-8dbd-45d5-a24e-1b3266c55936 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.720359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 225/459] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:28.720804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f30cf3-8dbd-45d5-a24e-1b3266c55936 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.722379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 235/460] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:35:28.727806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.728331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.728853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.741131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f30cf3-8dbd-45d5-a24e-1b3266c55936 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:28.765039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:28.767277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.809259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:28.816124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:28.896634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 226/461] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:28.904633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f30cf3-8dbd-45d5-a24e-1b3266c55936 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:28.957546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:28.958417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.958856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.959258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.975398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:28.977084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:28.994955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:28.996067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:28.997086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:28.997086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:28.997086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:28.998714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 236/462] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:29.004566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.040336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.184200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.192486 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecf69e-7f02-4088-9065-deba3954bebc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.201823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb945aa-139c-429d-ac2e-5e6282ffff8e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.235372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.236381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.236547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.236807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.255038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.257053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.266130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.266826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.267225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.267606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.268310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:29.268674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.270290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:29.275527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:29.276419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:29.277830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:29.277830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:29.277830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:29.277830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 237/463] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:29.280318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.287808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 227/464] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:29.288703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.312187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.396751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618a775-199d-41c0-8d1f-b7b876d22d06 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.400829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f320d-e941-40b9-a2dc-3afc6287d076 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.430647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.431342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.431696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.431971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.439206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.439794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.440069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.440331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.441014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:29.441312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.442663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.443259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.444746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.453332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.459644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 228/465] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:29.459644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-ca6a8ffc-5edb-483c-b7a7-be2690376adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:29.464651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:29.465722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:29.467136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:29.467136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:29.467136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:29.467136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 238/466] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:29.467536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd6b6-cb72-47e2-9725-aeef236e33ac None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.467817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd6b6-cb72-47e2-9725-aeef236e33ac None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.468086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebd6b6-cb72-47e2-9725-aeef236e33ac None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.473264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 239/467] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:29.474051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebd6b6-cb72-47e2-9725-aeef236e33ac None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.546888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.572356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f6c7-a8e1-440e-b139-daea771c7165 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.589342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebd6b6-cb72-47e2-9725-aeef236e33ac None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.608558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.609171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.609443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.609722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.610355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.612092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.632525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.633381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.633659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.633921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.643861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 229/468] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:29 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:29.644055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.644134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.644228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a98d4-5a75-4daf-a809-5e89ed10eb33 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.665797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 240/469] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:29 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:29.667549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed12dd6-87fc-4a5c-ac6c-1c8f011a930a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.667766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed12dd6-87fc-4a5c-ac6c-1c8f011a930a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.670838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed12dd6-87fc-4a5c-ac6c-1c8f011a930a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.712713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6ed12dd6-87fc-4a5c-ac6c-1c8f011a930a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:29.713868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6ed12dd6-87fc-4a5c-ac6c-1c8f011a930a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:29.784630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.811449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 241/470] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:29.813193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3f24c-41e4-4d20-86b2-d1bade8bffa2 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.813712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3f24c-41e4-4d20-86b2-d1bade8bffa2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.814180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3f24c-41e4-4d20-86b2-d1bade8bffa2 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.816029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 242/471] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:29.821491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.821779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.822032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.833872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e202c0-bcf6-464e-9622-442a2fae1968 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:29.851519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:29.852749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:29.877571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:29.884558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:29.885478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:29.886126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:29.886588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:29.887402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:29.887895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:29.889970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:29.906114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 230/472] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:29.913518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:29.940603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 243/473] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:30.039598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20082f8-77ec-4941-b550-05c3a07083cf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.081532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.082142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.082417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.082683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.083317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.084860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.112863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 231/474] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:29 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:30.134903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.253803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cba62-7f02-4291-9cb1-8c780a63f222 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.273164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.274324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:30.309224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006198-89c7-486a-8526-a46366c4503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.311700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006198-89c7-486a-8526-a46366c4503d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.312225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006198-89c7-486a-8526-a46366c4503d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.312939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006198-89c7-486a-8526-a46366c4503d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.336641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006198-89c7-486a-8526-a46366c4503d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.338473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006198-89c7-486a-8526-a46366c4503d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-09006198-89c7-486a-8526-a46366c4503d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:30.355313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:30.356422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:30.357828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:30.357828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:30.357828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:30.360009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 244/475] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:30.365578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006198-89c7-486a-8526-a46366c4503d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.375828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:30.429704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c7b3b-1e49-4e55-9f80-7078ab64beb4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.470114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.470780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.471060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.471331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.472037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.473460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.482948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.sql.core [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:35:30.482948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:35:30.482948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [parameters: {'id': 'fa37438119e940bba8d27c17112c10de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:35:30.482948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87987) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:35:30.496066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006198-89c7-486a-8526-a46366c4503d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result.read() Aug 24 17:35:30.501626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:35:30.502463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application yield resource Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise self.value Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application session.commit() Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:35:30.503455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:35:30.504882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:35:30.506891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result.read() Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:35:30.517241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [parameters: {'id': 'fa37438119e940bba8d27c17112c10de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:30.519136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:35:30.527832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.532579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 232/476] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 17:35:30 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:30.532579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.545864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.546801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.547303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.547781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.549146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:30.549702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.551630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.574135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 245/477] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:30.579562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.623244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:30.647814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567d6c-df4e-4438-bb5c-7e739ce59dc7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.670829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:30.719628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7227e-b3dc-4bad-b4ce-6c2bb93257d4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.754696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.755453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.755759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.756030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.758020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.758827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.759292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.759958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.760805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.762422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.784796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:30.788211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 246/478] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:30 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-60922b15-3f1b-4a7e-ad25-cf78244f7129 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:30.790354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:30.791274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:35:30.792142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:30.796082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 233/479] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:35:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:30.801517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:30.900494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:30.918765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec643-a34a-446d-b95f-531dfe5e33af None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:30.956868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:30.957883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:30.958356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:30.958893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:30.959878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:30.960435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:30.962213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe26ee-64d6-4b7a-b1ce-61c63c10ac17 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:30.972810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 247/480] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:31.978837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a143-dc7e-4a95-a42c-10e4e57392c7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:31.979365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a143-dc7e-4a95-a42c-10e4e57392c7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:31.979365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1a143-dc7e-4a95-a42c-10e4e57392c7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:31.980975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 234/481] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:31.987076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d74-b198-4cb9-a69f-8af6a260669e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:31.987501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d74-b198-4cb9-a69f-8af6a260669e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:31.987673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8d74-b198-4cb9-a69f-8af6a260669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.012417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-037f8d74-b198-4cb9-a69f-8af6a260669e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:32.014135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-037f8d74-b198-4cb9-a69f-8af6a260669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:32.101126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 248/482] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:32.125558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543088a6-e014-4d22-87d1-4a645b4c92fc None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:32.125830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543088a6-e014-4d22-87d1-4a645b4c92fc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:32.126094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543088a6-e014-4d22-87d1-4a645b4c92fc None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.127587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 235/483] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:32.134033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:32.134348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:32.134631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.156410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:32.157588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:32.235664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 249/484] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:32.587352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543088a6-e014-4d22-87d1-4a645b4c92fc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:32.635932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:32.649887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:32.721825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543088a6-e014-4d22-87d1-4a645b4c92fc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:32.783043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:32.783623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:32.783899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:32.784167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.804395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:32.813659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:32.814727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:32.815538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 236/485] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:32.821743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74bfa-03e2-49b2-9264-e5165aa9770b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:32.824036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:32.862933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:32.863644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:32.863926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:32.864482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:32.865312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.878117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e9765f40-da78-427b-a053-8ca80e356477 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:32.883439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:32.884528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:32.885497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:32.887899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 250/486] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:32.889890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cade93-a34d-4e23-b419-b43427c09cf0 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:32.890345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cade93-a34d-4e23-b419-b43427c09cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:32.890782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cade93-a34d-4e23-b419-b43427c09cf0 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:32.892567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 251/487] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:32.894461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cade93-a34d-4e23-b419-b43427c09cf0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:32.967066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:32.968619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9d010-7d5e-49a1-8d10-2daf990302b6 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.006380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.007042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.007360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.007700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.008356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:33.008656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.010034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75779e4-6100-4f4c-9939-7040db121225 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.019313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 237/488] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:33.020082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cade93-a34d-4e23-b419-b43427c09cf0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.020651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.020933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.021197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.045641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:33.047987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:33.061418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.061870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.062143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.062406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.063069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:33.063586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.065009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.076566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 252/489] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:33.078241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.140866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 238/490] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:33.141730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.167983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.191945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0935c-bcb5-46f2-a532-6034933beb0d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.219693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.227831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.229274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.229623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.229899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.242167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.243739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.253094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50b3d-0cb5-4336-81a6-206ce850fd7c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:33.258752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:33.259668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:33.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:33.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:33.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:33.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 253/491] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:33.270872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.293049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.294009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.295072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.295560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.306310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.308124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e3b7632d-dbfe-44a8-8d4d-23654245e0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:33.324913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:33.326479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:33.327302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:33.327302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:33.327302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:33.333813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 239/492] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:33.335334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bffc-ac2c-4187-bbd2-92eea49c9aa2 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.335833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bffc-ac2c-4187-bbd2-92eea49c9aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.336292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bffc-ac2c-4187-bbd2-92eea49c9aa2 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.338406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 240/493] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:33.339438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894bffc-ac2c-4187-bbd2-92eea49c9aa2 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.377100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b6210-d4b9-49b4-a519-8d108119d1b7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.412116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.412513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.412788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.413056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.413682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:33.413978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.415347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabcace-bbe0-4973-b3c1-76582b8d97db None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.425366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.428250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 254/494] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:33.429659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.429935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.430204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.448579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:33.449822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:33.455166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894bffc-ac2c-4187-bbd2-92eea49c9aa2 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.486655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.492461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.493114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.493338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.493600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.494236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:33.494533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.495934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.512376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 241/495] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:33.512376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.527060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 255/496] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:33.531542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.640802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b0ded-3ba8-466a-9a54-0df0dbc2eb03 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.643593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd9f0-348c-44d6-a96b-2c0d828aa266 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.676588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.677218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.677497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.677769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.678144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.678832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.679198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.679668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.682898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.688316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.689885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:33.690087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:33.694719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:33.695627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 256/497] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:33.700277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-27a1457a-6a8a-44e1-a8b8-0492aa45d938 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:33.703460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:33.704347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:33.705230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:33.707610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 242/498] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:33.709007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514f7bb-42f4-472e-9650-800af1b2b7a5 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.709498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514f7bb-42f4-472e-9650-800af1b2b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.709778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514f7bb-42f4-472e-9650-800af1b2b7a5 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.715013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 243/499] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:33.715013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514f7bb-42f4-472e-9650-800af1b2b7a5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.735555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.810470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c0e41-e82d-455c-aebd-7bdba1b6a394 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.819792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514f7bb-42f4-472e-9650-800af1b2b7a5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.844062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.844686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.844963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.845229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] PATH_INFO: `/v3/users/glance` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.852002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.853465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.853811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:33.854412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.854688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.854954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.855625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:33.855925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:33.857929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:33.866509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 244/500] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-5dfcf68e-6ab3-45df-926f-bc34d9a42ded None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:33.867058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:33.867964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:33.868880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:33.869381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.869449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 257/501] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:33.869746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:33.870027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:33.870297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:33.889232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:33.890455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:33.930682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:33.957042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 258/502] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:33.958245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:33.973953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21429c37-c571-4a3b-a2ea-859780dee0b3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:33.980684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.020839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.021501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.021726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.021995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.023143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:34.023768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.025734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.068734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47d5c-2a3c-4389-a63e-ea8e888b5e9c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.073363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 245/503] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:34.074192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.108719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.109344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.109614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.109882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.120477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.121912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:34.136740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:34.137639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:34.139161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:34.139161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:34.139161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:34.139161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 259/504] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:34.139161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.190492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002ee09-e2dd-46a0-ad40-ffe5558222c4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.210252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.213006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.231239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.232565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.232756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.233080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.243264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.244765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.245500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146657-0eaa-4fe3-95ce-f477e12e6c69 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:34.259212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:34.260241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:34.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:34.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:34.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:34.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 246/505] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:34.261307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.283890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.284495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.284774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.285041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.285764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:34.286065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.287499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.322917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 260/506] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:34.323767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.362381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8daca4-7b51-49c7-8a56-9b2a631191b4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.395962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.396574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.396853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.397124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.426792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c941c1-44c9-4193-8968-fb6b3eb04515 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.431386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.456000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.462034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.462638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.462918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.463192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.463863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:34.464168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.464394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.465550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.465884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.476858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 261/507] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:34.477676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.566625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 247/508] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:34.567472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.580159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70b688-e8a2-4a52-ad15-120965ae441a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.613867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.614489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.614764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.615032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.625511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:34.630639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:34.631742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 262/509] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:34.632558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.639550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.674773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11aa7-1f34-42d5-a730-f4bd98870d77 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.681590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.707807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.708350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.708634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.708904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.709537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:34.709839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.711228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.722684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 248/510] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:34.723466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.737607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e78277-9922-48b3-b01c-f0fcc8d57dc3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.771399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.772029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.772303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.772569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.823120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b6e7e-0352-499a-894b-b3d67ec2cad3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:34.837430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.838878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.848463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.857319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:34.857924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:34.858205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:34.858479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:34.859114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:34.860578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:34.881116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 249/511] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:34 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:34.881919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:34.888947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:34.947838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 263/512] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:34.948796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.018103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770b81-44a3-4aa7-a11f-51a16f4b3f8c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.060133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.061682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.062095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.062664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.066798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359320-929c-459b-8846-99eb18c0e3b0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.074948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.077067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:35.092565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:35.093518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:35.094367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:35.094367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.094367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:35.094367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 250/513] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:35.094607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.105036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.105642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.105914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.106182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.106864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:35.107163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.109016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.119525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 264/514] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:35.119525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.134953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.139936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.212921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391038b6-0f18-460f-bbd1-051bfaab3783 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.228485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbdf21-c3ba-4e85-bfa5-a84958e22fb7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.254579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.255221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.255525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.255806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.263794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.264017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.264644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.264919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.265186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.265297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:35.276809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:35.277688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:35.278669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:35.279746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.279818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 251/515] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:35.279818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.283801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:35.299764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:35.300696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:35.301568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:35.301568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.301568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:35.301568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 265/516] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:35.302351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.373027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.387294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.393689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8594300-97ad-4caf-b1d0-73b4a70b4f6c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.417127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced80b2-2682-4abb-8e6c-c5d6c0003959 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.429570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.430182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.430490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.430762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.431398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:35.431734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.433116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.445139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 252/517] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:35.445841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.456865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.457514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.457774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.458048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.469852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.471278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:35.486386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:35.487278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:35.488168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:35.488168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:35.488168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:35.492318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 266/518] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:35.492917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.556954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff210ba4-643d-4ece-bfbf-3ec898f2c52a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.592414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.593017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.593300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.593563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.594275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:35.594824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.596038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.603881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e080541-f995-4761-ae84-f7aa04f6cb6e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.620776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 253/519] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:35.620776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.631438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.634774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.655763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.656368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.656644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.656907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.657539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:35.657834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.659187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.677179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 267/520] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:35.677179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.728217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c5b5-3a59-48fe-b872-bdb4ff746532 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.764396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.765026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.765303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.765572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.772211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.773654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.779743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311d3fb-4be8-4586-ba93-5f6807348463 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:35.785200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:35.786235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:35.787137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 254/521] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:35.787647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.820914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.821439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.821688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.821951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.822591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:35.822863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.824252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:35.835464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 268/522] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:35.836106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:35.867767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.875455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:35.896993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c25cb-6438-44e4-a5d3-f4a1c0563ae2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.933570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.934183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.934470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.934738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.945750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179a3b-a8f8-4c7a-bba3-56871a89412e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:35.980701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:35.981414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:35.982286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:35.982626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:35.989780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:35.991276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:36.004497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:36.005617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:36.006477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 269/523] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:36.007092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.018952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.020683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.102251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.114937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.120903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1218bc-e511-4479-be5f-ab1b64100e61 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.131420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 255/524] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:36.132589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.159066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.159738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.160023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.160296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.160979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.161284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.162708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.181837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 270/525] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.182489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.245126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472e006-a64d-402c-a181-418358a257b8 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.283617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.284244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.284517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.284783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.292810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.294904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.295866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049acee2-4d36-4e92-80c6-8e801747f9f8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:36.306459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:36.307366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 256/526] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:36.308378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.337909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.338516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.338795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.339157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.339808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.340111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.342098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.345596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.359621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 271/527] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.359621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.375237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.424495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f90602-92d0-4c26-8ddb-a6d51f735740 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.460053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.460689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.460972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.461246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.461660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af17eb-f502-463d-ba7e-a489fca5de8d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.472643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.474135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:36.488994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:36.489915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:36.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:36.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:36.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:36.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 257/528] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:36.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.502222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.502831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.503113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.503388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.504146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.504452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.505893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.519508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 272/529] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.519508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.617004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.638887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c47fb-5d62-483b-ab40-db9dc2edd4b1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.640253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.664864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35972d-cf70-4a20-983a-ebf62a1be968 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.687281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.687945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.688329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.688608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.689454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.689766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.691318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.699860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.700995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.701285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.701559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.702284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=2ae693a209514a79b82bc3f9ba50dccb) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.702586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.704053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.715517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 273/530] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/role_assignments?user.id=2ae693a209514a79b82bc3f9ba50dccb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.715517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.717523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 258/531] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.723778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.824570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bd8a0-0691-4ae0-95ad-2adf7a703241 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.864559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.865181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.865459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.865728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.866380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:36.866683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:36.868386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e69575-8459-4ae9-8950-d6a52570d45e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:36.881900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 274/532] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:36.887845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690a99-57c8-4870-9512-78568d16d534 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.888713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690a99-57c8-4870-9512-78568d16d534 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.889002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690a99-57c8-4870-9512-78568d16d534 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.890763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 275/533] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:36.891639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5690a99-57c8-4870-9512-78568d16d534 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:36.895881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:36.907058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af97ea-9dc5-4ef0-9b4a-491879ac5b65 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:36.946018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:36.946635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:36.946913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:36.947186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:36.966948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.012325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5690a99-57c8-4870-9512-78568d16d534 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.028569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.030866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.050119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.050753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.051030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.051302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.058945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.061264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-c4b877d6-23c3-4c59-8515-7cd6103187d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:37.074020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:37.074922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 276/534] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:37.076977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.077574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.077574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.103467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:37.105073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:37.158557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 259/535] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:37.160175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.181530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.193855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 277/536] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:37.194942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.214549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.274311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67b6cb-a0e9-46dc-9806-27824ca73633 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.309896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.310513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.310794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.311070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.311797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:37.312100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.313560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.332257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 260/537] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:37.338046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.381451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefcf0d-6f00-422e-8f21-fee817b19261 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.440820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33026-9696-4dac-93cf-cdb3e34e7b33 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.443087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.456221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.456871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.457151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.457421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.473711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.478298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.478907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.479187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.479462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.480216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=7e690af7e92f48efb0cbc9dee2c87ba4) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:37.480526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.480802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.482560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.488658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 261/538] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/role_assignments?user.id=7e690af7e92f48efb0cbc9dee2c87ba4&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:37.509202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:37.510169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:37.510998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:37.510998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:37.510998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:37.510998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 278/539] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:37.517254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.526662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.583953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.689695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d3a69-8b27-480c-9d05-17d26d8c1c05 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.711732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdae-e9df-4e6f-9a04-70d48320ee82 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.730661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.731325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.731684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.732469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.746904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:37.752890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:37.753778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:37.754574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 279/540] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:37.762318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.772175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.773320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.773781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.774276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.775097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:37.775683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.777505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:37.807011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 262/541] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:37.814076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:37.871003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.905156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff10e7-1259-4bcc-8a2e-68bf1a64ea3a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:37.914225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:37.990805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:37.991430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:37.991730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:37.992003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:37.992822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:37.993131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:37.994521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.008564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 280/542] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:38.009776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.065204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0074c87-212e-4f69-8e16-36ddfee8abe1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.124280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.124910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.125184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.125453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.135131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.136609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.143671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db2ad-9653-4f68-a4d2-929518cd1af7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:38.155122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:38.156201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:38.157247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:38.160766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 263/543] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:38.164923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.205761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.206391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.206665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.206931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.222280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.223752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:38.245693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:38.246596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:38.247501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:38.247501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:38.247501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:38.247683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 281/544] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:38.253470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.294268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:38.319398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:38.343945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b2030-3e5c-4f2b-9d68-074bce99365d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.391917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ed4d2-9598-41ea-b438-d2409c563d91 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.420206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.420828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.421113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.421378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.422072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:38.422374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.423905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.426794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.427407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.427781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.428057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.428698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:38.428999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.430371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.440727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 264/545] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:38.443702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 282/546] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:38.451181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.452445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.603831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:38.604353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac71a-544d-4b81-b4a2-b5f1b09b83ea None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.648637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad450d33-2adb-4fa0-8f33-3a17800a6745 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.674023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.674677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.674959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.675235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.691195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.692695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.706941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.707584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.707862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.708127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.708850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c18cb9acd43548ed9c8efe69f77b8df3) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:38.709153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.710601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e45db-9f86-48c6-a116-7054c744fcdb None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:38.717737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:38.718650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:38.720741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 265/547] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:38.722278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 283/548] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/role_assignments?user.id=c18cb9acd43548ed9c8efe69f77b8df3&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:38.730200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:38.743800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:38.749491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8f395-b0f9-4138-b98c-6581bd99783a None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.749792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8f395-b0f9-4138-b98c-6581bd99783a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.750040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8f395-b0f9-4138-b98c-6581bd99783a None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.751816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 284/549] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:38.760973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.761306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.761514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.800226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:38.801383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:38.897856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fe54b-f13c-459f-bcaa-e89356e96462 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:38.926414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 285/550] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:38.934557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:38.935772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:38.936185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:38.936524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:38.937313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:38.937683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:38.939182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339cd8-ad8a-4465-9704-26c944f5a180 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:38.957520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 266/551] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:38.963305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.057575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.094285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19e28c-ef0b-4c71-99f4-351d713be61b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.108291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e1e21-2b59-4dbc-b032-fe7975303a97 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.108587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e1e21-2b59-4dbc-b032-fe7975303a97 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.108831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e1e21-2b59-4dbc-b032-fe7975303a97 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.110428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 267/552] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:35:39.115512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.115791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.116065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.137154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:39.139551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:39.144136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.144761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.145047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.145319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.207870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.215843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 268/553] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:39.278909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:39.280490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:39.388269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.414474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 286/554] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:39.420647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.438057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.546883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9419-5fc8-4e01-9afd-58271872f4ea None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.589654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f2b8-e6a6-4a25-ac80-ca42566ae493 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.598405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.599060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.599339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.599641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.620959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:39.625517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.626151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.626422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.626688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:39.634103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:39.636709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:39.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 287/555] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:39.645172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:39.645172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:39.645457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:39.656089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:39.657982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:39.659228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:39.659228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:39.659228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:39.659228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 269/556] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:39.662789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.719930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.726170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.768093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9353-f8f7-4d6a-b96a-79f85dc6b311 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.785826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd09a-2864-42af-93b1-d41c70d80950 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.804613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.805217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.805610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.805924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.806532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:39.806831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:39.808353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:39.818789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 288/557] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:39.819710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.825309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.825949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.826228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.826498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.827143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:39.827443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:39.828844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071e8b4-73ed-40b9-ba14-1bae2421b074 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:39.845077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 270/558] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:39.848789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94c7-707f-4f80-8709-84511e8885b0 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.849078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94c7-707f-4f80-8709-84511e8885b0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.849408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94c7-707f-4f80-8709-84511e8885b0 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.850672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 271/559] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:39.851371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a94c7-707f-4f80-8709-84511e8885b0 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:39.925713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.932447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc374fc6-9bfc-4f99-8a71-231f67c4dd5c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.966694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a94c7-707f-4f80-8709-84511e8885b0 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:39.968222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:39.968854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:39.969394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:39.969394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:39.977961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:39.980245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:39.986388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:39.987329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 289/560] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:39.988214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.010361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.010991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.011264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.011561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.022012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.023496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-aa73a94d-49af-4eec-b73e-6b4714264953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:40.038006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:40.039224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:40.040175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:40.040175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:40.040175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:40.040175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 272/561] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:40.040400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.040496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.040766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.058172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:40.059351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:40.107770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed73e-13d5-47a9-8ec1-435ed68356db None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.132645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 273/562] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:40.133712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.149833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.150587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.150934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.151284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.158311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.159884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:40.173633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:40.174494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:40.175449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 290/563] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:40.176560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.198037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.261492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.272428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023bd5-812c-44ad-9b37-e774479fe21b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.297031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03bf04-7527-4696-98f4-aa98c8f6838d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.310068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.311002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.311509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.311988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.312878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:40.313382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.314971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.327814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 274/564] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:40.328952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.336066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.336685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.336966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.337239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.337883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:40.338186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.340022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.358172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 291/565] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:40.359083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.441333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1af2-d785-4e99-828a-2b26c25b9ed7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.446356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.473577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305bb8-240a-487b-927e-b2a480cc6366 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.477117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.477728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.478009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.478278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.478960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:40.479265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.480752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.494108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 275/566] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:40.499714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.510052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.510940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.511220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.511520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.515249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.523789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.525340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:40.547344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:40.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:40.549638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:40.549638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:40.549638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:40.549638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 292/567] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:40.549952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.617530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf994c-0a16-4a1c-bb82-c34bde087415 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.656785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.657472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.657768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.658028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.660009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e6bd7-7fff-4cee-921e-b95b6e4ee4e9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.665586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.667037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:40.679866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:40.680840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:40.681792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 276/568] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:40.682879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.694044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.708396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.709032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.709307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.709570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.719739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:40.724836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:40.725797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 293/569] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:40.726678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.768237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.802621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53e3c1-3a08-40bd-9ac4-0f625f32c97e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.834658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f593ba-358e-4f0a-bb34-cd9feab00e1e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:40.838567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.839170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.839448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.839753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.840476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=ee2ab73d700247e5b5bd72ee331bb4fd) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:40.840782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.842870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.848373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 277/570] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/role_assignments?user.id=ee2ab73d700247e5b5bd72ee331bb4fd&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:40.849191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.879700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:40.880329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:40.880609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:40.880881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:40.881540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:40.881843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:40.883278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:40.896182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 294/571] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:40.897076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:40.953158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:40.973166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3df806-a743-4386-a817-9dd9ceed26f6 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.012630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.013770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.014045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.014313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.015028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:41.015137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a664-d5db-42c0-8e55-c56f11d89404 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.015328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.016818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77345de7-d3d8-4782-9c53-65ca7f38aae3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.019373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.034949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 278/572] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:41.036532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16951fd-ee2b-41e9-bff2-365e8b60e2ab None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.037086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16951fd-ee2b-41e9-bff2-365e8b60e2ab None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.037560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16951fd-ee2b-41e9-bff2-365e8b60e2ab None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.039299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 279/573] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:41.040237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16951fd-ee2b-41e9-bff2-365e8b60e2ab None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.058238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.058845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.059133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.059399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.060231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:41.060533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.061895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.070093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 295/574] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:41.070719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.162560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16951fd-ee2b-41e9-bff2-365e8b60e2ab None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.173294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c54cb-9224-4e4c-9796-06f8730f3dbe None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.183511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.202495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.203126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.203457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.203925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.208492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.209114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.209398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.209652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.211278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.212774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.221213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.222639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-45e2f42a-dbba-4800-97f0-4ded395966e7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:41.226023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:41.226957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:41.228315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 280/575] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:41.228937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.228937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.229356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:41.243228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:41.244437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:41.245340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:41.245340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.245340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:41.245340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 296/576] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:41.245561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.253701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:41.255320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:41.320728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.343080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 281/577] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:41.343959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.359657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9812977-1e3d-4332-8c0c-b8b8dda815c0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.406089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.407086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.407771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.408187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.423229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.424897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:41.444642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:41.445822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:41.446945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:41.446945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.446945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:41.454061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 297/578] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:41.454061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.464100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad413-6236-4cc9-b50a-7811c71df22f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.475948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.503930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.504696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.505076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.505473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.506362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:41.506770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.508831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.536226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 282/579] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:41.537510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.593192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.595345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab275cc4-fd8f-4e37-bd36-c770c6a5c7eb None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.638760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.639726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.639846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.640204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.640916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:41.641284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.643069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.659842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 298/580] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:41.659842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.664430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c296c01-3fec-4474-850c-13dde5c814f2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.706909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.707815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.708192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.708506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.709171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:41.709470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.710851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.727243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 283/581] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:41.727243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.739707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.768288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093d24-043e-49f1-9700-2352bbf9a4d8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.815599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.816369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.816650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.816922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.831234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:41.833673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:41.835305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d31b0a-3aee-4f58-8e92-359ae46089d2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:41.836997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:41.854862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:41.855936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:41.857413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:41.857413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:41.857413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:41.857413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 299/582] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:41.857413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.876831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:41.877517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:41.877797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:41.878066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:41.888412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:41.893082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:41.894184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 284/583] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:41.895218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:41.982607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a5702-1a4a-407b-92a6-981954eb0418 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.004539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc115297-2630-45a7-826b-50cc1ad5b15f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.009722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.019713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.020351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.020620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.020885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.027576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.029563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.040519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:42.041240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:42.042110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:42.043016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 300/584] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:42.043742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.044428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.044785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.045125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.054919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.056584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:42.068973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:42.070086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:42.071241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:42.072321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 285/585] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:42.073200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.098592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.161212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2200-282d-4f95-8665-bf34fd44b9cd None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.181038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55270ca-4fce-45db-94c3-0b2532f72b02 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.196464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.197062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.197335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.197601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.198281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.198516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.199963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.212709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 301/586] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.217745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.217970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.218583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.218860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.219134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.219824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.220130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.221513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.229651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 286/587] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.234906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.269003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.322087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.327652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef30aa-5fa6-4e82-b528-cfd57c8c29bf None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.341482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51499586-945f-4cca-8897-2114d05bb480 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.367926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.368628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.368811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.369073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.369758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.370057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.371492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.377871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.378762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.379040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.379308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.380054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.380353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.381755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.389110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 302/588] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.389110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.405855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 287/589] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.410750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.495262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2481f-bb9f-4d88-a256-fba5c7adf2a3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.502867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.532659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916af158-2357-4b53-bea4-b0bc0c1014a0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.543373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.544076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.544361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.544635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.552055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.553498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:42.564608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:42.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:42.566754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:42.570319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 303/590] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:42.571158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.572339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.573139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.573609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.573885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.584975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.586414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.588853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:42.604066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:42.605201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:42.606082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:42.606082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:42.606082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:42.606082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 288/591] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:42.606376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.682505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf851a1-fcbe-4c5c-bb39-78e078402bcc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.718888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.719731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.720081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.720421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.728380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109f73d-5b8e-4450-bf66-f0910d0fbc1d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.736313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=1a6e9e58daeb47abb6277e7c2bf6bdcb, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.737836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.752649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 304/592] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:42 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:42.753650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.773877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.781875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.782979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.783267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.783854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.799098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.800711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:42.817794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:42.818812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:42.819743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:42.819743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:42.819743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:42.819743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 289/593] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:42.819743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.875282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:42.878123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c07e-c572-41a2-8450-668326c98a2c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.922057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.923020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.923298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.923615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.924297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.924594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.926038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2f3c-bfee-4908-96c9-7cf6cd4cfee5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.936981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def69d8-c3d3-44a3-b913-5b9dab2dfa01 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:42.939371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 305/594] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.940529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3752-c153-4cc1-a38d-2c73d72ed387 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.940814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3752-c153-4cc1-a38d-2c73d72ed387 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.941080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3752-c153-4cc1-a38d-2c73d72ed387 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.942388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 306/595] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:42.942956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d3752-c153-4cc1-a38d-2c73d72ed387 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:42.978878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:42.979503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:42.979853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:42.980155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:42.980792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:42.981091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:42.983062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:42.995739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 290/596] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:42.996363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.042248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.069808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d3752-c153-4cc1-a38d-2c73d72ed387 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.098811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785b8c0-8afa-4bca-9b77-ed0aa123a2d2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.106787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.107082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.107385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.107705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.107973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.108784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:43.108906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.110258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf38c4b-a4eb-4ee5-a3f7-d7347aa82405 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.124119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 307/597] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:43.125305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.125582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.125845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.142860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.143552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.143693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:43.143864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.144108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/2ae693a209514a79b82bc3f9ba50dccb/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.144850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:43.159812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=2ae693a209514a79b82bc3f9ba50dccb, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.161318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.174861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 291/598] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:42 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/2ae693a209514a79b82bc3f9ba50dccb/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:43.180450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.228001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 308/599] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:43.229107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.316614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565107-a55d-4be3-86ea-6b0ed96a9b46 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.367773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.372013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.373545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.373901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.374134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.390568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.392073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:43.421215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:43.422343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:43.423303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:43.423879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f2eeb-30ed-4268-9949-f82d876d8b68 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.424279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 292/600] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:43.432288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.458785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.459456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.459858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.460381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.467818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.469341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:43.480929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:43.483917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:43.486668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:43.487727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.487806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 309/601] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:43.497051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.607300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71049365-8f9c-4a3d-b7fc-7df1687df2a1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.614326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0792d-6b4c-4820-b8e6-b2595155c944 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.642272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.642884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.643161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.643430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.644180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:43.644477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.645916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.650386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.656033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.657216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.657703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.658250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.659136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:43.659673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.661498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.664801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 293/602] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:43.665442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.676916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 310/603] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:43.678290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.742455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.774977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6dc879-0d51-458c-88a5-0d1645a669ff None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.792791 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c297d-b97c-4856-ab39-ed73eba17d20 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:43.816484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.817125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.817397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.817661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.829440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:43.834583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:43.835833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:43.836687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 294/604] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:43.837402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.838507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:43.839414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:43.839919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:43.840388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:43.852666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:43.854407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:43.870391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:43.871330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:43.872331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:43.872331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:43.872331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:43.874363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 311/605] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:43.875436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:43.920406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:43.973384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c8369-eabc-47d8-b5fe-8025c2499882 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.001939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936a7cc-deb6-4998-92ed-433e250462a7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.018034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.018844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.019204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.019614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/7e690af7e92f48efb0cbc9dee2c87ba4/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.022285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.036818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=7e690af7e92f48efb0cbc9dee2c87ba4, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.038410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.048534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.049395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.049861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.050273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.051052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:44.051537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.053018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.061690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 295/606] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:43 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/7e690af7e92f48efb0cbc9dee2c87ba4/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:44.062773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.064199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 312/607] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:44.069645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.233744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.238828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd9d69-5811-4f63-b470-80f143bab6f2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.274930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed4944-43a2-4f5c-be63-d7deb47a87cb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.281583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.282263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.282541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.282807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.297258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.298830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:44.317712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:44.318965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:44.320106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:44.320106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:44.320106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:44.320106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 313/608] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:44.321360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.321987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.322339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.322618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.323359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:44.323603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.324428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.324985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.343512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 296/609] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:44.348983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.417882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.444596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d77800-62f0-4cce-9fdd-4ba987261714 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.483167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.483943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.484173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.484443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.485081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:44.485380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.487009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.500198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 314/610] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:44.505217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.527848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.554257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60569-4380-43a2-898c-bf71a4b9fd5f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.623155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d078b-7093-4bb7-a75c-d3ecbbb818fc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.661863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.662498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.662775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.663049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.672734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.673390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.673730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.674006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.676851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.678311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.691073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.692576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:44.704572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:44.705518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:44.706437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:44.708420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 315/611] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:44.713557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:44.719972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:44.720843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:44.722708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 297/612] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:44.726785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.846503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9f545-542a-4c96-b43d-557c3b0459d2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.849602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.876509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:44.894232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.894862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.895145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.895421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.896145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:44.896454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.902946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959755eb-be02-4e6e-ab1b-567a68253388 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.920012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 298/613] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:44.921270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859db-ffcf-4398-9307-c8fc3b877fc2 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.921554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859db-ffcf-4398-9307-c8fc3b877fc2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.927005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660859db-ffcf-4398-9307-c8fc3b877fc2 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.927937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ab6c-6b4e-4aab-94ef-3a65e67f4d75 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:44.928675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 299/614] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:44.929415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660859db-ffcf-4398-9307-c8fc3b877fc2 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:44.971449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:44.972089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.972371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.972647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:44.973375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:44.973682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:44.975402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36958e-6cc0-44d0-9e8c-d9dbfd6cac28 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:44.996597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 316/615] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:35:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:44.998643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:44.999030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:44.999312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.018950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:45.020301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:45.039711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660859db-ffcf-4398-9307-c8fc3b877fc2 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.076165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.076832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.077113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.077389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c18cb9acd43548ed9c8efe69f77b8df3/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.094200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=c18cb9acd43548ed9c8efe69f77b8df3, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.094949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 317/616] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:45.095615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.095804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.110225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 300/617] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:44 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c18cb9acd43548ed9c8efe69f77b8df3/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:45.174503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.221183 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123673f3-57aa-4cf7-944e-b03baed77b13 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.263822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.264494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.264771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.265112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.283703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.285262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f29e7618-4157-4c68-a94c-b170b052e1a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:45.301814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:45.304251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:45.306369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:45.306369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:45.306369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:45.306369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 318/618] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:45.308834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.332026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2129a94-84e7-4f34-a60b-a046599cec5f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.332339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2129a94-84e7-4f34-a60b-a046599cec5f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.332572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2129a94-84e7-4f34-a60b-a046599cec5f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.334188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 319/619] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:35:45.341722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.341986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.342256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.349365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.364618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:45.365751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:45.444783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 320/620] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:45.445576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.450832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5748a-b7ae-427a-abf3-cdf1c455e2ff None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.488468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.489079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.489356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.489628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.490388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:45.490691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.492171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.504799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 301/621] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:45.510501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.537530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.600254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e84d7-3e9f-41a5-bbb7-937f25b2d62b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.630087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.647610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.648323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.648606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.648874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.661234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a4d87-bfce-44b8-a917-91ba0205559d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.665236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:45.670125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:45.671075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:45.676294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 321/622] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:45.691712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.706079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.707225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.707733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.708372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.715744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.718274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:45.730006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:45.730900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:45.731834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:45.738778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 302/623] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:45.740209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.816762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475caf85-7239-405f-8891-1f005414ce4e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.836472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.848429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01f49a-48c9-4834-8fc2-d383daa6b156 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:45.873777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.875013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.875706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.876152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.877225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:45.878081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.880219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.885514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:45.886228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:45.886767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:45.886939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:45.887786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:45.888183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:45.889715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:45.890775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 322/624] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:45.891798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:45.896122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:45.914184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 303/625] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:45.915158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.024292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ff561-c58a-40b2-87f6-700dc15b02bf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.028109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87efbc-5581-4351-84dc-bc02cedb661d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.061289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.062146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.062516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.062879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.072666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.073619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.074145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.074582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.074719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.077012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.087765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:46.092141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:46.093261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:46.094345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:46.094345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:46.094345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:46.095497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:46.096395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:46.097254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:46.099310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 323/626] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:46.100317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.101871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 304/627] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:46.103163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.132943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.163864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.220497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60aeb3d-f355-4ec7-9458-bcb3208db643 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.256262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef74a31-c03c-4e55-8a64-44c9e70aa569 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.277312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.279297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.280092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.280994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/ee2ab73d700247e5b5bd72ee331bb4fd/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.300178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.301170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.301522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.301863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.302586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:46.302952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.304734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.310963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=ee2ab73d700247e5b5bd72ee331bb4fd, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.313922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.323516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 305/628] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:46.323516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.351443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 324/629] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:35:46 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/ee2ab73d700247e5b5bd72ee331bb4fd/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:46.353888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.443022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0091f0-344c-488d-9eb8-8f3baf1a005c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.448396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.486132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.487100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.487100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.487338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.487989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:46.488258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.489657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.505485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 306/630] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:46.513411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.562816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.614598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faf66-1204-435b-b420-6c2bfa80d41d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.683805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.684443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.685251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.685518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.695620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.697305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:46.715060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:46.718505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:46.722079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 325/631] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:46.732725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.736217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2ac90-d781-46f5-9eeb-d3b19dd83336 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.788717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.789666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.790184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.790660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.804126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.805859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:46.823340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:46.824355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:46.825283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:46.825283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:46.825283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:46.827643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 307/632] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:46.832896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.864453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.898506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e24b6-331f-4508-88d3-166e5c27ca47 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:46.937340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:46.938080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.938361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.938726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.940217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:46.940217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:46.941463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4d2d0-b622-4815-813c-773e7d7c13e0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:46.943394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:46.967661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 326/633] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:46.969161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697e60-9f65-4612-bdbe-483d3f904820 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:46.969497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697e60-9f65-4612-bdbe-483d3f904820 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:46.969850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697e60-9f65-4612-bdbe-483d3f904820 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:46.971740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 327/634] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:46.972275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a697e60-9f65-4612-bdbe-483d3f904820 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:46.983526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9aef4-5068-4a47-a936-d9d5e9791dae None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.035694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.036932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.037397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.037856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.039298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:47.039806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.041900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39d223-7c87-4742-80bd-4f2e4539e39b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.059368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 308/635] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:47.060964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.061417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.061855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.086004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:47.087359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:47.094085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a697e60-9f65-4612-bdbe-483d3f904820 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.134251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.135218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.135505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.135803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.168109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 309/636] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:47.170174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.191674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.211879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.221097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.222812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496b0ab-d4b5-4380-9a23-922a7b8fde32 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.326373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8016d3-5ba5-4192-a103-f962c0fa9e6c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.352981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 328/637] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:47.354545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cccf45-7b0c-4e47-bed6-d026b156c8bc None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.354830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cccf45-7b0c-4e47-bed6-d026b156c8bc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.355102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cccf45-7b0c-4e47-bed6-d026b156c8bc None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.359543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 329/638] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:47.360516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cccf45-7b0c-4e47-bed6-d026b156c8bc None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.367864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.368790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.369288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.369758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.378070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.379834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-76a679df-c41d-45db-9d12-62ee1ce54850 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:47.392177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:47.393115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:35:47.394116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:47.396333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 310/639] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:47.397795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.398272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.398722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.416969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:47.418169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:47.487521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.494193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.499373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cccf45-7b0c-4e47-bed6-d026b156c8bc None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.500243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 311/640] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:47.501308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.548092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.549271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.550594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.551058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.565112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.568030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:47.589309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:47.590296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:47.591243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:47.591243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:47.591243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:47.594714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 330/641] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:47.595881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.628384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c0e52-bf5e-4cd5-b28f-52ad2d92f260 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.666245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.666960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.667312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.667716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.668653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:47.669620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.671229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.695292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 312/642] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:47.696051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.734509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eee8a7-7cc4-413f-9c8b-3dabefd083a8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.755927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.779960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.781103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.781944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.782705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.794292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:47.799959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:47.801051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:47.801915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:47.802563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77061ed-44ec-4522-9706-c6266211399a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.802649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 331/643] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:47.802649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.807292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:47.844071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.844924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.845376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.845788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.846624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:47.847064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:47.848590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:47.861631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 313/644] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:47.862514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:47.924429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659977db-0495-474f-a69c-c3057ec10337 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.963200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602f4-b347-4f50-937e-aabafce50273 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:47.965524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:47.966842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:47.967118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:47.967389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:47.984091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.002744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.003381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.003684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.003953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.013846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:48.018894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:48.020012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:48.020980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 314/645] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:48.021695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.049750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.051720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.054747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.126623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ca7f-fd3d-4f51-9727-8fd01c06bf2c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.160612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.161233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.161539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.161810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.162464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.162766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.164462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.170278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 332/646] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:48.170278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.177668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 315/647] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.177668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.223334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.277860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb1636-8a87-4c80-9512-66dc274a5fde None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.282025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.287347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4009a-8988-4693-be90-97d12fac9dcd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.316143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.316772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.317044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.317303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.324036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.324466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.325201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.326047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.326159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.326430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.327150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.327551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.328971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.337466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 316/648] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:48.337600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:48.338734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:48.339923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:48.343041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.343592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 333/649] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:48.344352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.447285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ac281-bcbd-4b49-bfb0-03ec625af51b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.451190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52bacd-d1cd-4a4c-b010-fccd4bb7663f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.452756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.481075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.482222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.482491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.482753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.488908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a753-699e-4286-81f0-40227839c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.489627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a753-699e-4286-81f0-40227839c967 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.489908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a753-699e-4286-81f0-40227839c967 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.490181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a753-699e-4286-81f0-40227839c967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.493770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.495270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.501909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672a753-699e-4286-81f0-40227839c967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.503408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672a753-699e-4286-81f0-40227839c967 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:48.511559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:48.513220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:48.514426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:48.514426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.514426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:48.514426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 334/650] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:48.514426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-2672a753-699e-4286-81f0-40227839c967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:48.522268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:48.523408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:48.524471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:48.524471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.524471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:48.524471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 317/651] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:48.524471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a753-699e-4286-81f0-40227839c967 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.538747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.629809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3127f-c847-4b2b-8a53-35d774c5c23a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.633232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a753-699e-4286-81f0-40227839c967 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.663877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.664497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.664771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.665040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.666993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.667639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.667909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.668175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.668873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.669170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.670603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.675442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.676936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.682113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 318/652] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.683008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:48.694891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:48.696082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:48.696965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:48.696965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:48.696965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:48.696965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 335/653] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:48.696965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.700956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.768137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.786163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543352e-accc-4948-a2bb-f7f316d77e93 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.801296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05e919-0f02-4985-816e-fed5593a249d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.818352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.818951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.819224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.819507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.820443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.820443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.821764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe8b15-d2ea-4f73-b0b5-341d00b8fe34 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.833235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 319/654] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.834568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687d32-b40e-49a0-b76f-590054475ccd None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.834845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687d32-b40e-49a0-b76f-590054475ccd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.835128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687d32-b40e-49a0-b76f-590054475ccd None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.836408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 320/655] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:48.837101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97687d32-b40e-49a0-b76f-590054475ccd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.838718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.839325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.839647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.839912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.841394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.842543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.843953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.857564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 336/656] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.858360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.913022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.938117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97687d32-b40e-49a0-b76f-590054475ccd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.955720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e277df-a220-4988-976c-fd96e7250ca0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:48.970956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.971590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.971867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.972159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.972872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.973176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:48.974597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:48.980378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 321/657] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:48.981174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:48.982562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:48.996892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:48.998054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:48.998328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:48.998592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:48.999248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:48.999571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.000972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7c202-c993-4627-896b-91aeff42b201 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.019325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 337/658] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:49.019325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.019325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.019325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.037164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:49.038324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:49.119696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 338/659] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:49.119696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.119999 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720e77-0a37-429f-a4ce-2f86e2f4d92e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.157742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.158653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.158936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.159211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.169763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.170755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.172673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:49.184228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:49.185711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:35:49.186631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:49.190727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 322/660] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:49.191134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.253861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db3f7d-350f-42c3-9e3b-61a178c50d89 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.274433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.298909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.299630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.299925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.300202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.310159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.311578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6387fa-b9ba-4cb4-8db6-c80573e21717 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.311711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:49.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:49.327923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:49.328831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 339/661] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:49.329331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.359872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.361562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.362286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.362754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.373933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.375755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:49.391527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:49.392531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:35:49.393552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:49.395934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 323/662] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:49.396861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.453431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1d761-5562-462d-9299-0aee78bb98b1 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.459705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.490079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.490735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.491009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.491272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.492509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:49.492805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.494652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.505204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591181f2-d04f-4c42-b3bf-f381f824d5f1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.521221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 340/663] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:49.522087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.545790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.546544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.546917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.547303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.548104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:49.548625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.550058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.560547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.568773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 324/664] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:49.569214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.633501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc498e-d2ba-473c-9abe-54b173501025 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.672274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.672907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.673185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.673455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.673884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e8081-fb74-48e9-b807-9a170b733927 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.683939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:49.689147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:49.690082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:49.690916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 341/665] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:49.692407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.698357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.717683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.718293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.718569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.718838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.719601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:49.719904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.721323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.741323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 325/666] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:49.742162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.801524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:49.809115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd293ba-7d37-40c9-ab7b-0c010cfa213e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.845208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.846020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.846299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.846570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.850012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405df3-4535-4d6e-acff-97b807c46566 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:49.887159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:49.888717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:49.889184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:49.889594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:49.890592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=2ae693a209514a79b82bc3f9ba50dccb) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:49.891035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.892732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:49.898832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 326/667] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/role_assignments?user.id=2ae693a209514a79b82bc3f9ba50dccb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:49.899899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:49.931584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:49.933060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.001375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.054883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.064875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 342/668] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:50.066034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.071988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8dce-c9a6-432e-802d-8e36f8a2ceab None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.114482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.115098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.115376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.115677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.116339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:50.116638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.118179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.132238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 327/669] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:50.132238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.199801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6d5fb-8c85-4225-b7be-1747ce6817a8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.255754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.256711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.256839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c123-9a36-4263-a9a8-c712e352f365 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.257297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.257729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.258980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.299022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.299943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.300321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.300733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.313166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.314630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:50.330545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:50.331861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:50.332974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:50.332974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.332974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:50.333908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 328/670] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:50.334897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.348173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.356167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.357897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.464023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc223d-4e5d-4665-beca-3f8fa0dcc1fc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.476305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 343/671] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:50.477869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.503205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.503954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.504304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.504654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.522295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.524514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:50.546021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:50.547028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:50.548574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:50.548574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.548574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:50.551799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 329/672] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:50.553078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.566658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.609001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.615967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1c199-2724-4c7d-8cdd-965390f72495 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.654418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.655168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.656054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.656054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.656694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:50.657105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.658625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.671641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 344/673] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:50.672688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.698287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f3973-6107-4473-bd3f-e074b81f36c9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.749084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.750113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.750828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.751917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.766310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.768850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:50.784468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:50.785455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:50.786384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:50.786384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:50.786384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:50.786384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 330/674] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:50.787324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.793287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222ab88-dede-4a7c-a214-eca97f26dfff None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.831085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.831783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.832133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.832483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.833206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:50.833806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.835041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.852832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 345/675] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:50.852832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:50.872763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.875394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:50.924227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce087cd-9164-447c-b72d-b066df1e507c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.961507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:50.962558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:50.962981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:50.963397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:50.966522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba73ea4-c04a-4efe-b7d8-b2250ac8708a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:50.971838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:50.973605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:50.986793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:50.987754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:50.988637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:50.990872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 331/676] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:50.991878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.007910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.008663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.008996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.009269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.009909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.010288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.011941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.025818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 346/677] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:51.026784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.104706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241daf63-036e-4ec1-a8c2-417d230a8041 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.109039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.125327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.133152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4d17c-b7f3-4e0c-87b8-9de7992fe45f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.145333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.146386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.146963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.147235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.155448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.157270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.167916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.168526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.168866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.169133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.169814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.170108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:51.170397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:51.171306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:51.177337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:51.178023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa600c4-ba60-41ac-9aa3-afb9ecb7ea83 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.178108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 332/678] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:51.178108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.195417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 347/679] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:51.196796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861c99-f2e8-4bf7-a833-f171f5eb1885 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.197086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861c99-f2e8-4bf7-a833-f171f5eb1885 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.197359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861c99-f2e8-4bf7-a833-f171f5eb1885 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.203441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 348/680] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:51.203441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861c99-f2e8-4bf7-a833-f171f5eb1885 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.286302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a3e6-2e4a-4d45-9e1d-d3806aaca9a0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.305837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861c99-f2e8-4bf7-a833-f171f5eb1885 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.321583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.322317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.322592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.322982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.329927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.331466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.339805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.340405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.341199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.341471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.342181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.342479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:51.343591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:51.344572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:51.345498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:51.346528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ab38d-b688-4094-b1d8-0041b31e8a51 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.347948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.349380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 333/681] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:51.350236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.370780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 349/682] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:51.370780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.370780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.370780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.386188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:51.387451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:51.395900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.458492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3106a1a-8309-44fe-81eb-ea710f53f8fc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.461656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 350/683] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:51.464474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.495784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.496396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.496670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.496986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.497710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=7e690af7e92f48efb0cbc9dee2c87ba4) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.498021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.499458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.511576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 334/684] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/role_assignments?user.id=7e690af7e92f48efb0cbc9dee2c87ba4&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:51.513630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.595611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8c7ad-1d4b-4d38-8c51-d99d03191b2a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.626868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.642197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.643174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.643675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.644162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.645070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.645575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.647869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.663281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 351/685] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:51.671153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.767245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786119a-8274-4dff-9b99-7a9433938fee None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.769169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:51.799823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaa37d-dc2e-40f8-b909-aed295b84ba9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:51.813044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.813939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.814219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.814485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.847392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:51.848151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:51.848428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:51.848695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:51.849768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c18cb9acd43548ed9c8efe69f77b8df3) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:51.850071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.852099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.860245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 352/686] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/role_assignments?user.id=c18cb9acd43548ed9c8efe69f77b8df3&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:51.864431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:51.933830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:51.935776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c75ae2-6057-475a-b401-dcb49cd3f248 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:51.968432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.044100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e442b-a4c7-4908-b914-fb77bb79015e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.083688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.084321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.084596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.084859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.096991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:52.097641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 335/687] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:52.099014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5213785-d51a-4c28-829d-a0be802de73a None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.099404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5213785-d51a-4c28-829d-a0be802de73a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.099864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5213785-d51a-4c28-829d-a0be802de73a None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.101364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 336/688] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d1fbf611-13dd-4d36-a94b-da6e5005a4ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:52.104134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:52.105077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:52.105921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 353/689] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:52.106938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.106938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.107509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.107632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5213785-d51a-4c28-829d-a0be802de73a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.110648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.127937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:52.129121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:52.209526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 354/690] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:52.210418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.259751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.260165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5213785-d51a-4c28-829d-a0be802de73a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.332794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508d21e-aa6a-4640-85ec-9842a7f6f52a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.360245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.362621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.364316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.365089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.381268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.382139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.383441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.383902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.385620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:52.386079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.389457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.391528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.393509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.407567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 355/691] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:52.413064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:52.429033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:52.430212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:52.431296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:52.431296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.431296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:52.439695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 337/692] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:52.456474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.533169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.577406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55299bf6-b4f1-4441-a0c2-0666152b0aa4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.597861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.601936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e5a2-11ce-41b7-8b1f-da93aa8410f7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.617008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.617615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.617890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.618157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.618892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:52.619194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.620589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.634920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 338/693] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:52.635861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.653715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.654938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.655690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.656170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.676399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.678623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:52.697837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:52.698831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:52.699760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:52.699760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.699760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:52.709195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 356/694] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:52.709195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.764104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7b53-7ff9-4239-9e39-26931a280f32 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.779507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.803975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.804873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.805872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.806293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.820853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.822889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.833231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6e02-7f52-45a1-a6ae-e163942cfc1d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:52.843771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:52.844723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:52.845614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:52.845614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:52.845614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:52.852604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 339/695] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:52.853634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.876681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:52.877614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:52.877614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:52.877899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:52.884971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:52.886489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:52.898171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:52.899041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 357/696] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:52.900422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:52.917367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:52.998372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6681a-473e-4f90-820c-25e1da6db8ab None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.017631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86895-8fd0-4685-8f41-0c9ecb045a30 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.036078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.036822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.037036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.037315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.037959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:53.038263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.039778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.057598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 340/697] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:53.057598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.066583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.067246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.067596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.067926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.068571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:53.068873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.070257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.087312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 358/698] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:53.087312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.095382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.175872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724728e3-bc4e-43d7-bf3e-5c683c4180ab None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.176078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.210738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2a9e-b99d-4f3e-8c89-45fa799a68af None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.220854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.221655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.222098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.222852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.223712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:53.224468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.226343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.242504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 341/699] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:53.243562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.261174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.262036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.262967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.263518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.271948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.273742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:53.287110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:53.288292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:53.289428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:53.295507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 359/700] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:35:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:53.295507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.359067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.378741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0bf14-03c1-41f5-8480-e4ee2af2e5ba None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.405132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f81ab-042d-4cc4-bdef-c608122e1c25 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.421679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.422557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.423683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.424217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.425964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.428525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.442046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.448140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.449918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.451078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.451190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.451877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=ee2ab73d700247e5b5bd72ee331bb4fd) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:53.452580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.454014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689e22b-e7a4-47d2-b5ca-09e837bc966b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.457368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 342/701] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:35:53 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:53.458554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.470671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 360/702] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:53 2022] GET /identity/v3/role_assignments?user.id=ee2ab73d700247e5b5bd72ee331bb4fd&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:53.637481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136660aa-1af9-43bc-9ca5-fc79ce3603b3 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.638191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136660aa-1af9-43bc-9ca5-fc79ce3603b3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.638450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136660aa-1af9-43bc-9ca5-fc79ce3603b3 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.640124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 361/703] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:35:53.645795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521230-d3ee-4b3f-99c8-696799c05917 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.646206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521230-d3ee-4b3f-99c8-696799c05917 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.646577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521230-d3ee-4b3f-99c8-696799c05917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.673848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9d4bd-10e4-4cd5-af19-c4b2ee2b8967 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.676456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ed521230-d3ee-4b3f-99c8-696799c05917 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:53.677683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ed521230-d3ee-4b3f-99c8-696799c05917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:53.721677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.722268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.722542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.722808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.723575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:53.723878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.725417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:53.742400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.752998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 343/704] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:35:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:53.754279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:53.767965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 362/705] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 17:35:53.869469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31457bf-53e3-4fc2-8cd0-e49228333a09 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:53.907203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:53.907772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:53.908093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:53.908371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:53.987797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:53.990933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:53.992780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c4a12-d804-4f77-a8ce-d87e1be31ddc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.056362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fb9-0669-4969-b367-5709f754c5b7 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.056701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fb9-0669-4969-b367-5709f754c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.056909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25fb9-0669-4969-b367-5709f754c5b7 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.058249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 363/706] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:35:54.061090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25fb9-0669-4969-b367-5709f754c5b7 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.116403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 344/707] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:54.122727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.123003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.123272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.142018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:54.143204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:54.173773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25fb9-0669-4969-b367-5709f754c5b7 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.202903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:54.219621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.220593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.221129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.221731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.222464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.222962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 345/708] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:54.223925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.224593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:54.239307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:54.240447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:35:54.241310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.249283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 364/709] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:54.250350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.289921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:54.395519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934095e-fd55-4eea-ab00-00712b501f5e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.429266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081caa-d6cc-4b48-a427-bd9042ad4d0f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.460019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.460672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.460950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.461219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.461837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:54.462132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.463507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.473955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 365/710] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:54.479408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.498289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.498974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.499252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.499535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.511234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.513283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:54.536624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:54.537618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:54.538524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:54.538524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:54.538524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:54.538524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 346/711] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:54.544397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.572253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:54.623281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ee175-00a2-4938-8516-5cbd8058753b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.640916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:54.662024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.662464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953c48f-e9bd-4c00-8d2c-be6a4f3299f0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.663050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.663541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.664075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.664944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:54.665404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.666963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.673962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 366/712] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:54.675029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.725323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.725939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.726291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.726475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.727113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:54.727411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.728920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.756749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 347/713] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:54.761545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.790715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d37e3-a527-42aa-980c-fe79e22d07bd None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.812251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:54.831256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:54.832204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:54.832666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:54.833091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] PATH_INFO: `/v3/services/network` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:54.833836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:54.835579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:54.846245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:54.847490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:35:54.848327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:54.855444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 367/714] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:54.857957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:54.958101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d0ea5-8737-4f2f-bc48-c743c092e263 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:54.973833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bdd2-a236-450e-9090-bd0931b23f76 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:55.016217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:55.017318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.017748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.018166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.018649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:55.019137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] RBAC: Adding query filter params (name=network) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:55.019504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.019782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:55.020006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.020376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.021627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:55.031524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:55.031524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:55.031856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 368/715] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:55.033958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:55.042707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:55.043707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:55.045041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 348/716] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:55.049962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:55.079894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:55.091244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:55.161130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d291b7-c5fd-4e27-be84-7246d62df649 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:55.161677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598afeef-99c6-4d17-8cce-cf6ca4926c71 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:55.197110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:55.197901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.198310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.198684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.199573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:55.200622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:55.203623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:55.214509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 349/717] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:55.220558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:55.245198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:55.245921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.246204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.246476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.247143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] RBAC: Adding query filter params (type=network) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:55.247441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:55.248839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981e883-0f52-4161-ad61-2f0375825c13 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:55.259726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 369/718] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:35:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:55.293516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:55.336345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037dd06-5e84-45c2-afe0-203124db6ce1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:55.375231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:55.376072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.376349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.376668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.377396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:55.377700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:55.379126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1372f-1516-4da6-a096-7fe4223df9fb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:55.385570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 350/719] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:55 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:55.914887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2746-b591-42c5-a1c6-7a37177dc1f5 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.915430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2746-b591-42c5-a1c6-7a37177dc1f5 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.915430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c2746-b591-42c5-a1c6-7a37177dc1f5 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.917050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 370/720] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:55.922643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:55.922931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:55.923205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:55.946978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:55.948209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:56.013281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11596f4b-c4f6-40f2-87fb-7d79fe7af76b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.013624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11596f4b-c4f6-40f2-87fb-7d79fe7af76b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.013820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11596f4b-c4f6-40f2-87fb-7d79fe7af76b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.015376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 371/721] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:56.020574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.020853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.021124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.051395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:56.052625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:56.056936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 351/722] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:56.152717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 372/723] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:35:56.435377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:56.492937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:56.511879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:56.545669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae83366-c19e-484b-af61-2f80f77bc458 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:56.580865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:56.581503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.581783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.582057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.593175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:56.597855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:56.598773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:35:56.599657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:56.603607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 352/724] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:56.605047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:56.632719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:56.711140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5168c-a718-47f7-9843-99a34d564afc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:56.733847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:56.748888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90644910-027c-4051-888b-063230e00906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:56.749509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90644910-027c-4051-888b-063230e00906 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.749793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90644910-027c-4051-888b-063230e00906 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.750069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90644910-027c-4051-888b-063230e00906 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.750743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90644910-027c-4051-888b-063230e00906 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:56.750927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cea4d-84e3-4238-bf64-c5823a3a00b2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:56.751028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90644910-027c-4051-888b-063230e00906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:56.752472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90644910-027c-4051-888b-063230e00906 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:56.763126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 353/725] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:56.775372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90644910-027c-4051-888b-063230e00906 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:56.829719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:56.830364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.830643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.830916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.852401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:56.864512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:56.865454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:56.866280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 373/726] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:56.871308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:56.910848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90644910-027c-4051-888b-063230e00906 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:56.973016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:56.974443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:56.974723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:56.974995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:56.995104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:56.998418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:57.022856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:57.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:57.024752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:57.024752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:57.024752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:57.024752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 354/727] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:35:57.033009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.071841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.077903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef56f-2fae-430f-8b3d-d4497af40a2b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.095107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.135033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.135680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.135960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.136231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.141686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:57.142005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.143412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.160680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 374/728] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:57.166364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.212387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699cccb-4fe6-42d0-a368-b39d8abc0438 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.264076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.265004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.265459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.265899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.267804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:57.268109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.269488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.299812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 355/729] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:57.300936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.341088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e780-0973-4581-a2b5-c33fe6d9eceb None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.383422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.384139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.384416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.384736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.396770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.398228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.411980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-96d2e96e-275f-4922-84e5-c09793c50d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:57.413025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:57.413965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:57.414912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:57.414912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:57.414912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:57.414912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 375/730] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:57.415846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97b58-4461-4ed4-a51f-2f0ce047cc73 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.417766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97b58-4461-4ed4-a51f-2f0ce047cc73 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.417766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc97b58-4461-4ed4-a51f-2f0ce047cc73 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.417766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 376/731] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:57.423331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc97b58-4461-4ed4-a51f-2f0ce047cc73 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.423497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a425-637f-4270-8795-325b584890e0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.461485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.462118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.462277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.462607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.462978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.470945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.472498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b4729c05-1218-4d58-9709-d524d504ab33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:57.486218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:57.487216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:35:57.488267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:57.492447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 356/732] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:57.493749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.494090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.494419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.513372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:57.514742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:57.566565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc97b58-4461-4ed4-a51f-2f0ce047cc73 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.604790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.605453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.605728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.605992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.606711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:57.607010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.608509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.611041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 357/733] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:57.612684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.633346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 377/734] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:57.638970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.714634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.745866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f088284-2766-4a52-8d16-6a9e70c91f50 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.755266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:57.784107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.784410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2eed-02cb-412f-9a18-8a94118cf38d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.784635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.784910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.785180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.785900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:57.786201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.787660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6316d-f367-407f-bf85-7e906d820c2c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.805117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 358/735] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:57.806154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712cb5-8858-477d-b468-72543fb29770 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.806437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712cb5-8858-477d-b468-72543fb29770 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.806707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712cb5-8858-477d-b468-72543fb29770 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 359/736] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553696-f6ef-4edb-81c8-0ee7ce376633 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553696-f6ef-4edb-81c8-0ee7ce376633 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553696-f6ef-4edb-81c8-0ee7ce376633 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 360/737] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:35:57.812372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e553696-f6ef-4edb-81c8-0ee7ce376633 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:57.831234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.832292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.832885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.833227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.841192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:57.842868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-ff568b85-30bd-4f45-a85d-c273ddbe8e61 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:57.854419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:57.855347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:57.857467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 378/738] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:57.858060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686cc-1e79-4800-a99c-a45be2ae73fc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.858060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686cc-1e79-4800-a99c-a45be2ae73fc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.858060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686cc-1e79-4800-a99c-a45be2ae73fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.875749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7a1686cc-1e79-4800-a99c-a45be2ae73fc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:57.876942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7a1686cc-1e79-4800-a99c-a45be2ae73fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:57.927636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e553696-f6ef-4edb-81c8-0ee7ce376633 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:57.945042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 379/739] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:57.946357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.946644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.946915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.962121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:57.962785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:57.963058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:57.963329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:57.964190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:57.965904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:57.973965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.027318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.044108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 380/740] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:35:58.044108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.054608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.056767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.147269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232e9a-c25c-460f-a985-3d882cd5fa1f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.182538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.183201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.183500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.183819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.185232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:58.185534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.186967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.189994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 361/741] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:58.190875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.205852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 381/742] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:58.206689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.234291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.265408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.301799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d1302-0b6c-4fc5-bc30-e96161b4c9f1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.317013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e6b37-d0be-41ed-9b71-76be96e4cd15 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.335668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.336272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.336548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.336812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.337444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.338800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.351487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.352135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.352409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.352680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.360307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 362/743] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:35:58 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:58.361597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.364157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.365627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:58.382744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:58.383857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:58.384762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:58.384762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:58.384762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:58.384762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 382/744] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:58.384762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.484149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.505955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66fe9-ac96-49fc-8790-76dd409d0e9f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.517815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ee4e-d0d5-416a-babc-8fe7ebfbd997 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.521212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.550883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.551554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.551892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.552114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.567306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.567970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.568247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.568522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.580832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:35:58.586004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:58.586962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:58.588204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 383/745] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:58.588899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.634548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.636104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.696243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83994645-2680-4fd5-bb67-5d3c3a67b93a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.734377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.734986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.735260 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.735559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.736457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.737823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.740975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.754438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 363/746] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:35:58.754438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.758686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 384/747] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:35:58 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:58.763104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.798952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:58.882884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cacd8b-f389-422d-b674-77adffcd256f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.916038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5d82-3da2-41a4-af03-441c8d6cab47 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:58.931766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.932530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.932747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.932987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.933628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:58.933931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.936573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.951252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 364/748] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:58.952122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:58.956120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:58.956736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:58.957011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:58.957277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:58.957929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:58.958231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:58.959636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:58.970180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 385/749] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:58.970833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.022496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.073452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.090806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80cc3c-dcf6-4176-a296-8390bbf197fa None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.093330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817bcdb-d12b-4f86-bdf6-1fb118daf5b6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.131961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.132689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.133090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.133369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.136990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.137877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.138338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.138815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.140970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.143216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.152014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.153728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:59.156998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:59.158198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:59.159637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:59.163055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 386/750] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:59.164014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:59.175245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:59.176250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:59.177145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:59.177145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:59.177145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:59.179833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 365/751] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:59.180890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.283776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130e221-6706-4742-9867-64af1a6d5aa0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.305786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54d5c-e3cd-4d18-b579-abe85f0ef18c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.313635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.319986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.320865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.321302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.321722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.333646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.335310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.337488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.346403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.347155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.347513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.347910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.348681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:59.349051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.350562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:35:59.353202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:59.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:35:59.355030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:35:59.355030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:35:59.355030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:59.361737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 387/752] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:59.363569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.368265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 366/753] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:59.368575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.473314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b241d-3d5f-4a3e-98ac-d54264f3e137 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.485067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8103-fa63-46bd-98df-5a1bad7c61c3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.513319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.514279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.514783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.515291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.516154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:59.516660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.518251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.531138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 388/754] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:59.532382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.536027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.537167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.537167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.537370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.537905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:59.538206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.539669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.557425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 367/755] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:59.557425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.583818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.598369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.645855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a6c0-7cda-4900-b135-2b4ca805e643 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.667651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163d33e-9524-421e-9ffc-8a74d4347ff5 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.682383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.683080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.683428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.683801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.684686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c78ca59d6dbd409ea40d1a9991c8263c) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:59.685000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.686532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.699552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 389/756] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/role_assignments?user.id=c78ca59d6dbd409ea40d1a9991c8263c&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:35:59.699552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.729282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.730338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.730984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.731638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.743573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.745542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:35:59.771548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:59.772558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:35:59.773447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:35:59.783017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 368/757] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:59.789056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.843112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d2ec5-9ecb-4a81-bd04-da8e5fef99a6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:59.878890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.906642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:59.907288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:59.907594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:59.907864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:59.916787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:59.918262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:35:59.932619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:35:59.933853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:35:59.935588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:35:59.939633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 390/758] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:35:59.940788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:59.958468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:59.977835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322f409-00f9-4c8c-925d-268112753216 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.041512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.042191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.042934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.043321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.044916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:00.045320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.047318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.079341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 369/759] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:00.087862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:00.139136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989acb-0b03-4e17-b402-97ed315feb16 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.236452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.237157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.237342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.237621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.238382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:00.238702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.239439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea8eb-41eb-4ca9-be76-719411aa1e9b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.240168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.283527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 391/760] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:35:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:00.286064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:00.319052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.319610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.319925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.320207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.320996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=ee2ab73d700247e5b5bd72ee331bb4fd) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:00.321307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.323151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:00.323962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207bceb-33fd-47ab-87f7-652ed3d77abc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.351523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 370/761] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/role_assignments?user.id=ee2ab73d700247e5b5bd72ee331bb4fd&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:00.394562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90800fa1-bf31-47e0-9c9f-2a4ded0cb7a3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.431751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.432485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.432849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.433185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.444910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.446482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:00.462599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:00.463630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:00.464537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:00.464537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:00.464537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:00.464537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 392/762] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:00.469964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139171-ae32-4ae9-b6e2-5527294a5919 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.470248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139171-ae32-4ae9-b6e2-5527294a5919 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.470519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139171-ae32-4ae9-b6e2-5527294a5919 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.472232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 371/763] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:00.473000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:00.477771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.478046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.478319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.499827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:00.501406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:00.510732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:00.609079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:00.612391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 372/764] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:00.621750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5430-7428-4773-afed-c130b0bfd97a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.667041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.667677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.667957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.668226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.668992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:00.669291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.670649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.691206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 393/765] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:00.696788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:00.843651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1cbea-c222-4b21-aece-66bdaa0121c7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:00.887097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:00.887709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.887995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.888271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.900895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:00.902373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:00.907744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-bec10cd0-178e-4fe7-9b5c-555eabd8bfe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:00.917447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:00.918352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:00.920710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:00.923605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 373/766] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:00.925230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:00.961174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff496-3490-4ab5-99bb-f74797f88773 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:00.961490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff496-3490-4ab5-99bb-f74797f88773 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:00.961650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff496-3490-4ab5-99bb-f74797f88773 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:00.963199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 374/767] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:00.964008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeff496-3490-4ab5-99bb-f74797f88773 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.055142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e46ff-4b22-45b4-be02-57850482eb92 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.098452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeff496-3490-4ab5-99bb-f74797f88773 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.103769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f8990-8699-4e41-9b39-37596060209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.104496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8990-8699-4e41-9b39-37596060209e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.104865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8990-8699-4e41-9b39-37596060209e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.105231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f8990-8699-4e41-9b39-37596060209e None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.106624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8990-8699-4e41-9b39-37596060209e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:01.106963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8990-8699-4e41-9b39-37596060209e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.109084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f8990-8699-4e41-9b39-37596060209e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.126357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 394/768] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:01.128191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.128608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.129012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.146636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.147561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.147990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.148409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.149215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.150783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.152998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:01.154575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:01.157393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:01.163721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:01.165046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:01.165890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.173241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 375/769] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 17:36:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:01.173241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.243560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 395/770] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:01.245987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.275615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:01.374906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695feed-228d-4f5d-b9ea-4a1214390b92 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.379120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e29941-d07b-48c0-ac64-82084d25f686 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.415838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.416366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.416645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.416930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.417539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:01.417840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.419920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.425036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 396/771] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:01.430714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.433376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:01.472207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.472896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.473178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.473453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.477578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=1a6e9e58daeb47abb6277e7c2bf6bdcb, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.479055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.508745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 376/772] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:01 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:01.556768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.575597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6aabc-71ef-40b1-839f-6ec186c3c293 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.619981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.620867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.620867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.621149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.623742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:01.623742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.624718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.632472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 397/773] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:01.646432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.727937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:01.750469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f483c6-a9a1-4515-8255-43bdaabcc80d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.761401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f57f95-4701-4aca-bd7b-6dff59443391 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.802779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:01.828486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.830623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.830968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.831245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] PATH_INFO: `/v3/services/network` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.832131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.833899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.835002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:01.836157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:01.837079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:01.837564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:01.838749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] RBAC: Adding query filter params (interface=public, service_id=6973e244375b4085a035874b301d0d64, region_id=RegionOne) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:01.839264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:01.841501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:01.844362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:01.845337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:01.846179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 377/774] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:01.854782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 398/775] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/endpoints?service_id=6973e244375b4085a035874b301d0d64&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:01.861011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.864161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:01.973599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac5306-3998-4154-b535-8401a50ae644 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:01.973799 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c088-5427-4326-a53b-e21c75d7b523 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:02.011605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.013087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.013574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.014066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.014893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] RBAC: Adding query filter params (name=network) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:02.015395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:02.017084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:02.022525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 378/776] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:02.027696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.029012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.029293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.029563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.029788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:02.032185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:02.033774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79390f4-1398-4fbb-bb67-ccd672334604 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:02.035760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:02.050760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 399/777] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:01 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:02.064963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e536ee-7844-44bf-9312-481a6ad77251 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.065303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e536ee-7844-44bf-9312-481a6ad77251 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.065523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e536ee-7844-44bf-9312-481a6ad77251 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.067143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 400/778] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:02.073082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.073396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.073634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.105729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:02.107590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:02.159545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:02.233078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 401/779] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:02.240751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e87233-4ed2-4dd5-9205-281970b6582d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:02.277867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.278459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.278740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.279013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.279727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] RBAC: Adding query filter params (type=network) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:02.280028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:02.281450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:02.287982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 379/780] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:02.293942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:02.418188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cd42b-0822-48a7-bc50-669ec14d6b0e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:02.442579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:02.454850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.455465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.455821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.456102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.456868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] RBAC: Adding query filter params (interface=public, service_id=74fe371ebaac4a8d804e363e1dc47d95, region_id=RegionOne) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:02.457177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:02.458623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:02.464455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 402/781] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:02 2022] GET /identity/v3/endpoints?service_id=74fe371ebaac4a8d804e363e1dc47d95&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:02.469976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:02.609883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3a57-c57f-41ee-9c07-342534b989ce None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:02.620910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:02.627984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:02.660189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.661104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.661603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.662054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.662928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:02.664612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4baa4-2f93-4ffd-ab83-ca126f9249da None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:02.671687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 380/782] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:02 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:02.673124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26939d7-e37d-4029-8267-efb60db28117 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.673611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26939d7-e37d-4029-8267-efb60db28117 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.674065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26939d7-e37d-4029-8267-efb60db28117 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.675824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 381/783] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:02.681525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.682017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.682495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.705014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:02.706584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:02.766563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837d4ff-490e-475e-91dc-41fa744ad2dc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:02.812208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:02.813410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:02.814820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:02.816120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:02.838340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:02.844458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:02.845439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:36:02.846287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:02.849117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 403/784] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:36:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:02.854434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:02.863796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 382/785] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:02.931852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:02.971518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77a1f6-7ae1-406f-8592-797ca4525d56 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.019512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.019512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.019512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.019512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.019935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:03.019935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:03.019935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:03.030330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 404/786] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:36:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:03.036409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.173310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d30cf7-b67d-4e7b-9c24-11935e1e0260 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.196219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.197699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:03.265545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.267468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.268121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.271046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.288177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:03.291000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:03.321937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:03.322967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:03.323899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:03.323899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:03.323899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:03.330450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 383/787] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:03.335100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.374464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08324421-4bc6-43c9-97dd-6c7474cddd0f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.405072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:03.507633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.508673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.509148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.509613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.565659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:03.579554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecb6c3-243c-48ab-a531-102ce9271105 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:03.580294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:03.581205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:03.582026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:03.591902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 405/788] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:03.592829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.628216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.628829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.629107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.629376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.630014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:03.630313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:03.631723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:03.657242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 384/789] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:03.666128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.718641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:03.780077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91cbe-c0ea-423f-a837-4f928da8199a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.830749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.831368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.831787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.832067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.832725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:03.833028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:03.834504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:03.844114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 406/790] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:03.849538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:03.882059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725ce4-65d5-4968-bcd6-e631c8dbbf9c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:03.907847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:03.943107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:03.943765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:03.944043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:03.944312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:03.955177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:03.956662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:03.976566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:03.977590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:03.978522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 385/791] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:03.984169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.040375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f2590-6c03-437a-a6c3-1c57ae109518 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.112225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.112982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.113268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.113540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.129253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.131679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.134475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:04.161505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:04.162482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:04.163360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:04.163360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:04.163360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:04.169854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbdf60-bc1e-4e60-b892-3e2f89cdaaad None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.172370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 407/792] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:04.173385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.207375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.208023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.208305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.208579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.209313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:04.209619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.211057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60822544-c1b8-4bee-9dd5-4d8018d83a02 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 386/793] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:36:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a545e8-3be4-4115-b869-0921fa091533 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a545e8-3be4-4115-b869-0921fa091533 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a545e8-3be4-4115-b869-0921fa091533 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 387/794] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:04.235749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a545e8-3be4-4115-b869-0921fa091533 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.291999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.300072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21ab0-a92c-4a03-8a64-b2f93f34c933 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.336627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.337467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.337979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.338459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.339315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:04.339840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.341661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238239cb-35e4-4592-8eb1-5a85a6192b26 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.354477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 408/795] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:04.356917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1446d3-678c-4529-8fc5-56beb2a24f69 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.357428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1446d3-678c-4529-8fc5-56beb2a24f69 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.357897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1446d3-678c-4529-8fc5-56beb2a24f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.364278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a545e8-3be4-4115-b869-0921fa091533 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.378313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ff1446d3-678c-4529-8fc5-56beb2a24f69 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:04.380172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ff1446d3-678c-4529-8fc5-56beb2a24f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:04.412009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.412655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.412957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.413470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.425167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.428886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.430828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:04.449143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:04.450320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:04.451209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:04.451209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:04.451209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:04.451209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 388/796] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:04.451209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.464586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 409/797] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:04.464586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970325-9506-4a8f-96b0-90e2fd40e878 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.464586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970325-9506-4a8f-96b0-90e2fd40e878 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.464586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970325-9506-4a8f-96b0-90e2fd40e878 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.465258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 410/798] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:04.466084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970325-9506-4a8f-96b0-90e2fd40e878 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.558770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.580410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef16b1-0ba5-47eb-903e-2829e24877f2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.586419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970325-9506-4a8f-96b0-90e2fd40e878 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.617130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.618054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.618474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.618950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.622757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.623444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.623821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.624167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.624879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:04.625252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.626696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65071dfe-a356-4ceb-8965-ff18f350332c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.629696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.631326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.643564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 411/799] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:04.643564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b827-937b-4483-8191-32a8880ea94b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.643564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b827-937b-4483-8191-32a8880ea94b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.643564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692b827-937b-4483-8191-32a8880ea94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:04.648992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:04.649907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:36:04.650950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:04.653196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 389/800] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:04.654173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.660765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e692b827-937b-4483-8191-32a8880ea94b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:04.662099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e692b827-937b-4483-8191-32a8880ea94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:04.714000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.739504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 412/801] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:04.739504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692b827-937b-4483-8191-32a8880ea94b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.783042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b5f7-5b89-468b-bb76-6734f166e649 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.816344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:04.819895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.820620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.821262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.822068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.830199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.831868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:04.844990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:04.845907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:04.846790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 390/802] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:04.847867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.865713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692b827-937b-4483-8191-32a8880ea94b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.910866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:04.911528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:04.912057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:04.912340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:04.913493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:04.914062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:04.916204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:04.936089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 413/803] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:04.936946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:04.974335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4e5d-cced-455b-82c1-d03d9d5b595c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:04.989750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.014146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.015271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.015727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.016336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.030956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:05.036855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:05.037988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:05.039046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:05.040948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 391/804] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:05.041882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.050876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aab6bf-21e2-4402-9a37-a7ef7d2fd54f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.076606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.087838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.088446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.088720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.088987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.089666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:05.089963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.092011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.103581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 414/805] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:05.104374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.150704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a9d68-20a9-4df5-b50b-cf2c8a03701b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.185071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.185765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.186046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.186316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.196840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.198292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.202847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8118-1900-441b-adee-de66d5538f5e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:05.212308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:05.213235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:05.214176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:05.214176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:05.214176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:05.214176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 392/806] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:05.214176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.238728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.239353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.239661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.239937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.240070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.240513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.241946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-684cc595-c890-4e35-b353-e222703bbeaa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:05.252415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:05.255697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:05.256549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 415/807] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:05.257342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d801c-0b11-482f-9f4d-5a40050f6384 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.257342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d801c-0b11-482f-9f4d-5a40050f6384 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.257342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d801c-0b11-482f-9f4d-5a40050f6384 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.257342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 416/808] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:05.257342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d801c-0b11-482f-9f4d-5a40050f6384 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.304634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.318884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2000bd5-e8ad-4998-8834-e9ed3b8bba29 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.352185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.352723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.353000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.353276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.353931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:05.354235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.355643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.360146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d801c-0b11-482f-9f4d-5a40050f6384 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.364496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 393/809] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:05.365284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.397945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.398625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.398903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.399174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c78ca59d6dbd409ea40d1a9991c8263c/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.415832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=c78ca59d6dbd409ea40d1a9991c8263c, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.417305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.431695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 417/810] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:05 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c78ca59d6dbd409ea40d1a9991c8263c/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:05.432571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.549871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.557186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791d4c-42eb-421f-96d2-ab3e0745c75a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.589465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.603420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.603985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.604261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.604528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.605154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:05.605260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdd093-3e3b-40de-aaff-e5535447f1c1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.605451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.606839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d7f55-5653-4fe4-b769-afc32dec4995 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.612023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 418/811] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:05.613197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.613478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.613751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.636690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:05.638438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:05.642733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.643358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.643677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.643939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.644578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:05.644879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.646686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.666131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 394/812] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:05.666131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.709621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 419/813] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:05.710812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.786625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67733-9a41-4e68-adb0-7379b6dc1bbf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.811313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.822542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bebf29b-a0f6-46d1-bc45-a15f448de8c0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:05.833712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.835273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.835887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.836438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.852259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.855014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.859206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:05.859857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:05.860131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:05.860400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:05.861350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:05.861650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:05.863029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:05.864901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:05.874443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 420/814] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:05.874443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:05.882289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:05.883274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:05.884195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:05.884195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:05.884195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:05.887744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 395/815] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:05.889109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:05.989721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3db27-1753-48d6-b59b-321f588a433b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.025979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.026606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.026881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.027150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.028726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a706-7b1b-454e-8582-3d30cc8301e5 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.034350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.035821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:06.047357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:06.048290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:06.049193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 421/816] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:06.049685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.065600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.069325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.069944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.070222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.070493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.071147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.072540 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.099029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 396/817] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:06.105388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.167201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51d104-9cdd-4ef0-9d13-69ed16e3224a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.193067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.217919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.218709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.219677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.220080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.220848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:06.221306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.223268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.243633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 422/818] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:06.244950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.250472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88457605-92ba-44e9-a907-47642ab406c0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.294760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.295406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.295713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.296360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] PATH_INFO: `/v3/services/network` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.296951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.298809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:06.312368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:06.313388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:06.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 397/819] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:06.315097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.331103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.373912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee98430-e4ef-44d2-9694-d8c26b723f0b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.409243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.409877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.410116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.410384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.411073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:06.411378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.412840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.425572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 423/820] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:06.426420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.439995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ccd1b-3872-44e0-abd2-72fa2412ebcc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.459355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.479659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.480616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.481728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.482233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.491178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.493051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:06.506118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:06.507298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:06.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:06.510469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 398/821] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:06.511420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.553873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304cb62-47d1-402c-84ab-4a89bdf76468 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.583744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.590756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.591386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.592172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.593045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.593625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] RBAC: Adding query filter params (name=network) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:06.594276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.595872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.600732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 424/822] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:06.601619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.634240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe0e3-e5bb-4f8a-939d-385e5e74131b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.672095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.673022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.673557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.673977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/ee2ab73d700247e5b5bd72ee331bb4fd/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.676180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=ee2ab73d700247e5b5bd72ee331bb4fd, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.677822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.693942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 399/823] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:06 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/ee2ab73d700247e5b5bd72ee331bb4fd/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:06.695331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.713297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374cc29-84ec-480e-8e00-705ebc90008a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.745256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:06.757787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.758747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.759173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.759689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.760623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:06.761135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.764146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.786576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 425/824] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:06.793095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:06.819052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f08591-3cd7-4590-a398-639fd750cc25 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:06.864053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:06.864803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:06.865147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:06.865486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:06.866200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] RBAC: Adding query filter params (type=network) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:06.866567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:06.868176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:06.881333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 400/825] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:06.889581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:07.010648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:07.051709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e3a04-09c2-4ffc-acc0-5de2c69e2d3a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:07.053703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:07.068651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaae7a-c928-4754-8adb-66edb796ee52 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:07.089212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:07.090742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.091217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.091745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.092609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:07.094297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:07.106778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:07.107610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.107836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.108110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.117197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 401/826] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:06 2022] POST /identity/v3/endpoints => generated 337 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:07.213538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:07.214948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f48d1-8ae0-4009-a057-6a2317ad429a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:07.280794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:07.333961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 426/827] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:07.339782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:07.450425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54d45-a7bc-4fd1-ab90-d5d598c42c70 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.450965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54d45-a7bc-4fd1-ab90-d5d598c42c70 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.450965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54d45-a7bc-4fd1-ab90-d5d598c42c70 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.452582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 427/828] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:07.457700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.457983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.458318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.463847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814d0cc-c152-4335-85a0-74f9496172f3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:07.484014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:07.485829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:07.509989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:07.510736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.511023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.511298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.526530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:07.530129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:07.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:07.553682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:07.554587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:07.554587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:07.554587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:07.556794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 402/829] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:07.565494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:07.578852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:07.595253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 428/830] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:07.692842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad9890-4b88-4419-b824-3310fa08cc45 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:07.728539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:07.729482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.729963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.730461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.731323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:07.731844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:07.733405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:07.746845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 403/831] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:07.752513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:07.842881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:07.863133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8682a2e-2f2d-47cf-8cd4-706a49f52c4b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:07.899906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:07.900602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:07.900875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:07.901147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:07.908794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:07.910243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:07.913426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:07.922881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:07.923882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:07.924788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 429/832] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:07.932660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.002342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.041583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb363077-21bb-4f5f-ae83-539d04d6082f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.053249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad432-1483-4884-b646-19d3c38b8fda None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.080191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.081198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.081680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.082158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.089495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.090229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.090940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.091219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.091944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:08.092243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.093675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.095702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:08.100659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:08.101599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:36:08.102445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:08.109477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 430/833] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:08.109630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 404/834] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:36:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:08.110737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.114018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.115071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.218096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee22df-ceed-48c3-ba59-bbbdc6e29baa None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.220715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b5a8-faf6-4b11-ba62-6bfd5de1dfb9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.248874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.255693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.255836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.256012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.256282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.256940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:08.257242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.258618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.260444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.261266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.261776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.262595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.263780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:08.264308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.265925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ddd8d-40ba-45f6-807a-eaae2cd57f7c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.272882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 431/835] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:08.273880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.278117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 405/836] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:08.351623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.401072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a41c-0f18-498a-969e-ce6841335f0e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.440922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.441561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.441841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.442107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.445640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8134-90ea-4dd2-95cc-8039c67aa1f4 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.445927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8134-90ea-4dd2-95cc-8039c67aa1f4 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.446199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8134-90ea-4dd2-95cc-8039c67aa1f4 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.448654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 406/837] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:08.453009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.453292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.453563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.454221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.455688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:08.470465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:08.471698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:08.472595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:08.472595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:08.472595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:08.472595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 432/838] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:08.475646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:08.477013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:08.477676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.576775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 407/839] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:08.604161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c895ef5-0157-4dcd-a917-7c102fb5f402 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.622441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.643081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.643946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.644372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.644823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.645605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:08.646052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.647588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.660247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 433/840] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:08.668714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.776382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a5904-0fa4-46fb-8d50-1cc81fec53c1 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.798606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:08.813885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.814521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:08.814800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:08.815073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:08.822539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:08.824037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:08.835885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:08.836838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:08.837861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 408/841] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:08.844586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.876846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:08.960427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e12cc-c6e0-4722-bd47-462473855821 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.997615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d65de6-37e0-4274-809e-fe3a1cca1a0d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:08.999040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:08.999859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.000289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.000774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.001503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:09.001806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.003227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24e21a-58ac-4190-998f-bb233ad8f93b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.026044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 434/842] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:09.026044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cdf9-da4e-4579-aff8-9d7a22c0ec5b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.026044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cdf9-da4e-4579-aff8-9d7a22c0ec5b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.026044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96cdf9-da4e-4579-aff8-9d7a22c0ec5b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.026044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 435/843] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:09.030251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96cdf9-da4e-4579-aff8-9d7a22c0ec5b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.040378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.041352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.041352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.041612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.042177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.044126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-5c75cc56-9874-49c2-9a4a-3576e289cbac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:09.053417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:09.054578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:09.055425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 409/844] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:09.056610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.056610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.056610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.073422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.074052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:09.075868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:09.118321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.151647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96cdf9-da4e-4579-aff8-9d7a22c0ec5b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.153214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 410/845] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:09.154236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.197918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.198830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.199356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.200275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.272697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.274221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.292812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56e7ea-a254-49f4-a565-e0add2f9e7a6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.311954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.330471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.331080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.331360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.331664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.332298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:09.332600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.334288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.339318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 411/846] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:09.345473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.385428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 436/847] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:09.390821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.403878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.471095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758032c5-c52a-47bb-8e7a-37599b4c349a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.507825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a136a-ebbf-4537-aec9-1ad90c2bd4b9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.513443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.514080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.514360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.514632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.515261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:09.515607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.519821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.531458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 412/848] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:09.531458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.552381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.552938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.553211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.553475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.555671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.566599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.568110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:09.584233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:09.585313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:09.586241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:09.586241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:09.586241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:09.586241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 437/849] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:09.593574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.658474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22297bb8-9953-4b94-aeb4-526bf5bf09bb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.675867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.698219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.699016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.700026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.700463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.712533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:09.718489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:09.719733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:09.720610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 413/850] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:09.725848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.729014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec01d6-83a0-42cc-88f4-f6c9f771643a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.765561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.766226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.766502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.766827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.767536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:09.767844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.769216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.784863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 438/851] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:09.790336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.841134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.866057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b3db-10b2-458b-bf86-049523db1094 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.910517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.911269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.911649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.912001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.912383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730939e-9fe4-4855-aed6-7f1a37ca3855 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:09.912799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:09.913175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.914939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.927562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 414/852] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:09.932423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:09.962605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:09.963389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:09.963743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:09.964269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:09.967978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:09.974624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:09.976140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:09.991365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:09.992418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:09.993389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:09.995549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 439/853] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:09.998745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.080352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48042f5f-3dc1-4efc-b776-70a0cf2b0e52 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.117777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.118770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.119255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.119783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.129424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0029b4-9f3f-463c-b9c8-4f29956ab889 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.131029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.132758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.138207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:10.149296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:10.150237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:10.151171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:10.151171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:10.151171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:10.153775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 415/854] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:10.159165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.173993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.174584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.174864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.175134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.175848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:10.176146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.177562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.195561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 440/855] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:10.196607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.261362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.290493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223e2a-f8fa-49db-a3a1-1e2b20ca9314 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.316897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d64b79-2809-45e0-8856-d87cd8324822 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.328804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.329703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.330187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.330743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.331716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:10.332222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.334006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.350123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 416/856] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:10.355956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.359858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.360736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.361075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.361350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.362073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c78ca59d6dbd409ea40d1a9991c8263c) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:10.362380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.363884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddb8e4-6ffc-4952-bdb6-f31d6900614f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.370313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 441/857] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/role_assignments?user.id=c78ca59d6dbd409ea40d1a9991c8263c&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:10.375532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf093-342a-41c4-8420-b634af3f17af None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.375532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf093-342a-41c4-8420-b634af3f17af None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.375532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bf093-342a-41c4-8420-b634af3f17af None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.375532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 442/858] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:10.378706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.378993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.379266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.400335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:10.401870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:10.439140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.488400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321063f3-90bf-4783-8a05-ccd322d8bf86 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.511519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 443/859] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:10.544813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.545535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.545881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.546218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.549839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.555302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.556859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:10.570799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:10.571824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:10.572759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 417/860] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:10.578279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.689144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebd5ad-3276-49ed-8be1-53ebc8895e81 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.724886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.725491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.725761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.726025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.726741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:10.727036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.728482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.749296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 444/861] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:10.756547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.798393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.818362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.876962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49ffa0-cd63-4781-bc1b-781df360ff7b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.917357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ad834-c7a9-44ef-a295-030d44337bad None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:10.920272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.920882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.921153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.927516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.967890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:10.968539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:10.969106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:10.970097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:10.970598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:10.972212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:10.984032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:10.985023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:10.986186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:10.989907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:10.993625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 445/862] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:10.994540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:10.999238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.000944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.080368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:11.101189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d27fa-2325-43ea-9cdb-ca9fc3607b0a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.114344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 418/863] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:11.123663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.136692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.137922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.138276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.138615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.139313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:11.139720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.141157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.150750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 446/864] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:11.151678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.243984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:11.250660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941f5d3-c007-406f-af71-03985b446c68 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.259854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75979-c4cf-4c63-a037-aaa0e6503a3d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.289710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.290836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.291266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.291715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.296327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.296939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.297218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.297488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.298115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:11.298417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.299820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.304458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.306062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.309989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 447/865] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:11.323166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:11.324481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:11.325551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:11.325551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:11.325551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:11.327449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 419/866] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:11.332971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.349489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:11.449472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b766c-ce23-457b-a6f8-47cecc3e6b8b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.488938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.489908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.490380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.490841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.491746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:11.492241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.493821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee3c19-cd50-4012-ae3d-08a62f433149 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.507427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 448/867] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:11.513835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.536700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d298ac-9108-48bd-b971-f8768336db9a None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.537230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d298ac-9108-48bd-b971-f8768336db9a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.537230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d298ac-9108-48bd-b971-f8768336db9a None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.538470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 449/868] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:11.544386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.544658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.544925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.564054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:11.565242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:11.641902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:11.648511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 450/869] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:11.652507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a77d-ce6a-4e9b-ad37-db965e501b38 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.690583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.691277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.691585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.691858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.699123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.701268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:11.713482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:11.714367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:11.715295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 420/870] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:11.720805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.828258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d6e14-15ed-4e5a-8366-4a137c196f03 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:11.862806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:11.863570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:11.863845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:11.864108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:11.864785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:11.865082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:11.866559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:11.879630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 451/871] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:11.885667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.897476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:11.971559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:11.991551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849cb12-c5a1-4903-8df1-8e228ebd1b95 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:12.026456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:12.027110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:12.028005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:12.028267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:12.028982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=ee2ab73d700247e5b5bd72ee331bb4fd) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:12.029285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:12.030707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edbcc3-1c16-49a1-98ab-804f2c3d3f31 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:12.035248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:12.037011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 421/872] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:11 2022] GET /identity/v3/role_assignments?user.id=ee2ab73d700247e5b5bd72ee331bb4fd&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:12.083897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202d6c8-f8bf-40a5-a1cd-dfe19231bad3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:12.123982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:12.124606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:12.124879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:12.125149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:12.125817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:12.127205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f675d7e-6320-4daa-8ab4-0bc6f690524a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:12.151680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 452/873] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:11 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:12.560153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d905-db81-4bc0-8592-01b94b529ff2 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:12.560483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d905-db81-4bc0-8592-01b94b529ff2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:12.560677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376d905-db81-4bc0-8592-01b94b529ff2 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:12.562274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 422/874] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:12.567561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:12.568093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:12.568093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:12.587083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:12.588304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:12.668939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 453/875] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:12.935633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d905-db81-4bc0-8592-01b94b529ff2 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.008114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.039166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376d905-db81-4bc0-8592-01b94b529ff2 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.075171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.075835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.076113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.076376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.087553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:13.092358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:13.094495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:13.096554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 423/876] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:13.104602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.154632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.209123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a05848-d2e5-4cbc-94a8-9ece76169a1a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.244416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.245027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.245300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.245565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.246263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:13.246559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.247949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267f2dd-f97a-42a8-a601-7b60d46f925d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.256579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 454/877] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:13.262559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.338119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc80d-d7e1-432b-a05b-735c573df99b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.338434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc80d-d7e1-432b-a05b-735c573df99b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.338656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dc80d-d7e1-432b-a05b-735c573df99b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.340906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 455/878] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:13.345553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.345842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.346118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.365823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:13.367978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:13.370066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaffb73c-c64a-44d3-955d-a0ebaec6e833 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.403103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.408923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.409758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.410148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.410486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.421820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.423344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:13.438248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:13.440219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:13.441950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:13.441950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:13.441950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:13.441950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 424/879] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:13.446287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.455112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 456/880] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:13.552325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e27fd-bd1e-40ff-beff-29498453f599 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.588334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.588956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.589240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.589512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.590685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:13.590989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.592394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.605148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 425/881] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:13.611163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.653571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.715365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb698a9-1009-4b33-b1be-8958a6e8671d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.728806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.752059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.753067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.753549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.754027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.761699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.763452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:13.775163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:13.776116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:13.777046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:13.780086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 457/882] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:13.785751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.787358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.850176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb8954-21b0-4881-89a8-972697ae295c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.888181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.888799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.889080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.889356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.890016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.890788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.893069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.895840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b86c7-9c0c-41b3-96c7-4dd54ab76b39 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 426/883] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:13 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:13.939040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.940068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.940539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.941000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.941925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:13.942398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.944072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75a55-15b3-4c77-ba32-5cf0be46dd5e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.965404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 458/884] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:13.971571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.055562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44303-b26f-4915-beb9-65ab74f3d0c8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.056176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44303-b26f-4915-beb9-65ab74f3d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.056691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a44303-b26f-4915-beb9-65ab74f3d0c8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.058567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 459/885] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:14.064336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497807-c195-4c12-9cee-4460b0983658 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.064895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497807-c195-4c12-9cee-4460b0983658 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.065161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497807-c195-4c12-9cee-4460b0983658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.085384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-22497807-c195-4c12-9cee-4460b0983658 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:14.086613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-22497807-c195-4c12-9cee-4460b0983658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:14.090112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817da6e-c0c8-4ba9-b409-74efdd94e812 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.093300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.137683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.138295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.138577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.138968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.176965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 460/886] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:14.218969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9a914-fe5f-4a5f-aeab-264af985174b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.219279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9a914-fe5f-4a5f-aeab-264af985174b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.219539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9a914-fe5f-4a5f-aeab-264af985174b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.221311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 461/887] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:14.226640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.226922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.227197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.227716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.229505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.249669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:14.250880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:14.342380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 462/888] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:14.347553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 427/889] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:14.352991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.368282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.460251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.464041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.465369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9617570-1f98-4f70-9d95-4b51d887de8e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.501387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.502031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.502310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.502582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.513686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.515143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:14.530083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:14.531037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:14.531986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:14.531986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:14.531986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:14.531986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 463/890] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:14.538027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.582257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa63c-cb4f-462c-9247-7a580ab881f7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.617091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.618446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.619296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.619803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.620268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.621586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.623161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:14.632696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:14.633704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:14.634527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.636493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 428/891] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:14.638434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.659332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6c0fa-e105-4773-9e17-3151f7a367db None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.692868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.693461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.693734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.694004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.694624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:14.694918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.696297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.708132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 464/892] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:14.708917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.725078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.766029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b53f-2dc9-45fc-82a4-19939b26480c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.808397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.809674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.809950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.810220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.811032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.811279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a81c1-bac3-4d41-ae11-07a075df230c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.813179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:14.824020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:14.825157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:14.826067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 429/893] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:14.827284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.841971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.856242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.856851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.857129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.857393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.858040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:14.858341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.859769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.869604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 465/894] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:14.869604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.938068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7887-3589-4faf-b631-3f6844643564 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.969899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:14.975456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5e8f6-7070-4681-8e43-f7ae004cb872 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.977122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.977770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.978052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.978326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.986296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.987767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:14.999946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:15.000979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:15.001882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 430/895] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:15.002369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.022659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.023270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.023652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.023926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.024542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.024841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.026195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.036340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 466/896] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.036340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.095009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:15.118407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554228c-ff03-48fd-98fa-d5e33fb87cf7 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.145921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae76791-5375-47e9-a3af-0b9fae694b65 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.154402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.155001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.155290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.155590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.156209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.156509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.157864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.167715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 431/897] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.167715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.188847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.189484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.189765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.190042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.190743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.191051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.192559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.208836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 467/898] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.225668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.252263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:15.319217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a76d91-b4de-4254-a93b-27aa76aa2816 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.357880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.358493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.358774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.359092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.359753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.360104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.361736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.366065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0465e2-653f-4dbd-8a4b-501feb05414d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.367055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 432/899] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.372844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.383871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:15.407963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.408500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.408772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.409044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.409766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] RBAC: Adding query filter params (role.id=ce6853f1894b4b67aeb6e1982c2c4008, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c78ca59d6dbd409ea40d1a9991c8263c) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.410074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.411528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.417460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 468/900] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/role_assignments?user.id=c78ca59d6dbd409ea40d1a9991c8263c&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=ce6853f1894b4b67aeb6e1982c2c4008 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.489665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7b2b5-88ba-4a6a-8081-1ee8b43322b0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.528336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.528962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.529241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.529513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.530226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] RBAC: Adding query filter params (interface=public, service_id=839e611e6fab4c9eab842a1da7793787, region_id=RegionOne) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:15.530531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.531967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b05455-5477-44d0-ad11-ffe0f59becdf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.534187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:15.546195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 433/901] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/endpoints?service_id=839e611e6fab4c9eab842a1da7793787&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:15.548445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:15.658224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793dbbd-fd2b-4e1f-9915-f4d8155d7f2f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:15.694461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:15.695073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.695359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.695661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.696272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:15.697640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaffb65-2187-476b-9dca-2a9280ab4622 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:15.704078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 469/902] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:15 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:15.926506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a192b1-1edb-4a18-9628-729da5a78c26 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.926759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a192b1-1edb-4a18-9628-729da5a78c26 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.926922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a192b1-1edb-4a18-9628-729da5a78c26 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.928528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 434/903] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:15.933977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:15.934270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:15.934568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:15.953014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:15.954224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:15.990992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:16.025774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 470/904] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:16.290466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a192b1-1edb-4a18-9628-729da5a78c26 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:16.395844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a192b1-1edb-4a18-9628-729da5a78c26 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:16.429863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:16.430488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:16.430758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:16.431021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:16.431629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:16.432988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:16.442311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:16.443286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:16.444844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 435/905] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 17:36:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:16.449584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:16.566975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500451a-5928-45ed-9cd2-e0af21e64d7e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:16.601587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:16.602204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:16.602485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:16.602758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:16.603384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:16.603722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:16.605093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:16.610937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 471/906] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:16.616294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:16.669215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:16.724430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5264ad5-5f44-40f6-9579-87fbdde02acc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:16.760669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:16.761279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:16.761576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:16.761846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:16.762473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:16.762768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:16.764161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:16.769737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 436/907] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:16.775080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:16.814350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:16.890012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf70f-4faa-44f5-902b-d0298d62ff02 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:16.926328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:16.927330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:16.927897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:16.928372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:16.929265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] RBAC: Adding query filter params (interface=public, service_id=4c0011e49cd84bcc927977252e562c6a, region_id=RegionOne) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:16.929772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:16.931423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b0e-7450-4f8e-85be-950d4bcb7d55 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:16.937790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 472/908] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:16 2022] GET /identity/v3/endpoints?service_id=4c0011e49cd84bcc927977252e562c6a&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:16.944903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:17.047048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8324-44a1-421d-baa5-662bbde88817 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:17.060171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:17.082090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:17.082690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.082965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.083228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.083851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:17.085211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633d01d-a8be-4d30-8827-8d1a5c28236e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:17.091757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 437/909] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:16 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:17.257361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2902ce-2f2c-4da9-ab22-579d63bd2149 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.257734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2902ce-2f2c-4da9-ab22-579d63bd2149 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.257914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2902ce-2f2c-4da9-ab22-579d63bd2149 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.259554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 473/910] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:17.265600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d145-a1d3-405d-a72b-4b7aa6aaa1e9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.265877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d145-a1d3-405d-a72b-4b7aa6aaa1e9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.266145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47d145-a1d3-405d-a72b-4b7aa6aaa1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.285732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fd47d145-a1d3-405d-a72b-4b7aa6aaa1e9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:17.286918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fd47d145-a1d3-405d-a72b-4b7aa6aaa1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:17.355810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 438/911] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:17.392409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680270bd-961c-4b46-b5a9-e06782fdde2e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.392409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680270bd-961c-4b46-b5a9-e06782fdde2e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.392409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680270bd-961c-4b46-b5a9-e06782fdde2e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.393122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 474/912] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:17.398579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.399115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.399622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.418237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:17.419792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:17.499583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 439/913] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:17.628679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680270bd-961c-4b46-b5a9-e06782fdde2e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:17.735060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:17.740289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680270bd-961c-4b46-b5a9-e06782fdde2e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:17.765420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:17.779607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:17.780213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.780407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.780674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.781308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:17.782778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839e614-5d5a-4716-82ce-f076381f100b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:17.801276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 475/914] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:17 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:17.802609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e2c28-efa5-430e-be61-c3dd16f2c9af None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.802948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e2c28-efa5-430e-be61-c3dd16f2c9af None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.803290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e2c28-efa5-430e-be61-c3dd16f2c9af None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.804961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 476/915] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:17.806217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:17.818014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.818350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.818562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.840515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:17.842100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:17.891424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df7918-cade-41a5-8bad-8fcf6613fa7f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:17.942541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 477/916] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:17.947087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:17.947756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:17.948048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:17.948301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:17.964796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:17.970370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:17.971399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:17.972266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:17.975559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 440/917] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:17.978301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.075941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.094117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae900ecb-742e-479c-8356-10254d060cd4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.132122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.133124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.133608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.134083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.134973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:18.135505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.137674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.146978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 478/918] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:18.152726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.224755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.267151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.278075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387bf1b-20f3-431b-b23f-14213e7ec4fd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.314768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.315396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.315855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.316138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.327748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.329124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:18.344483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:18.345459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:18.346364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:18.346364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:18.346364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:18.346364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 441/919] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:18.351292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.354467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe51af-de36-4dce-8034-9474e60097eb None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.403677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.404904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.405762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.406253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.407018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.408598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:18.422811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:18.423849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:36:18.424688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:18.425436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 479/920] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:18.430465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.449338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.472946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b2c74-f4f0-4e58-aaa3-6396d5a52cbd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.510053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.511011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.512061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.512529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.513449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:18.514042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.515665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.530157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 442/921] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:18.542727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.547427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.557095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10387a2e-6276-4b3c-80b7-e37748a8edc6 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.596208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.597043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.597583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.598035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.598902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:18.599344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.600884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.607100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 480/922] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:18.612608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.670053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165a928-5279-443a-868c-f2fe8335dea8 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.706142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.710067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.710770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.711173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.711645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.719779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.721314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:18.733879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:18.734830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:18.736390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:18.739921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 443/923] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:18.740504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c669-87e2-4f24-a601-116e607e36c3 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.740966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.779856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.781116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.781596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.782092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.782995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:18.783470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.785138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.791083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 481/924] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:18.796738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.815931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:18.868392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dafd9-b65b-4c71-a7f8-1a3a79741616 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.910912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.911557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.911841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.912118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.912963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:18.913276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.914823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.930993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d0d4-a36e-4707-ad11-2a4c82566edf None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:18.939003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 444/925] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:18.940349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:18.969948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:18.971113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:18.971669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:18.972152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:18.973068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:18.974832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94692d-a05b-4fec-a034-fb33bd4b2d2f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:18.986766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.011705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 482/926] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:19.070369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce555abf-e654-4a59-b24c-4a8413721d07 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.072538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa061ea-9f73-4f9e-8919-1d76f17c1406 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.072830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa061ea-9f73-4f9e-8919-1d76f17c1406 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.073104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa061ea-9f73-4f9e-8919-1d76f17c1406 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.075322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 483/927] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:19.081023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.081310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.081588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.099887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.101187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:19.102660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:19.124042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.124763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.125146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.125537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.140085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.141716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:19.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:19.162741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:19.164145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:19.164145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:19.164145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:19.164145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 445/928] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:19.170963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.194835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 484/929] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:19.282084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8cc5d-9b6c-4f58-9f69-b3de9855add9 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.318903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.319532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.319817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.320102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.320804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:19.321105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.322472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.335217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 446/930] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:19.340759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.447767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61ed48-a701-4333-96ba-2a0640a3e501 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.466801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.475683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.486448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.487581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.487866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.488142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.495734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.497220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.507662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:19.510656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:19.512100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:19.513556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 485/931] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:19.518060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.609456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4ff93-f552-4392-aa52-2cc51ce1a45a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.639844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0a79b-cc0a-4765-99bb-e064c415ee20 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.655257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.655964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.656251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.656510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.657293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.659002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:19.671012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:19.672046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:36:19.672956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:19.677264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 447/932] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:19.678104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.681615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.682477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.682961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.684017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.684978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:19.685501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.687133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.700568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 486/933] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:19.708554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.765827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.786950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:19.795946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc96219-cbdd-4e78-9439-e3ca64d57816 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.832397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.833440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.833917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.834381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.835365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:19.835940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.837492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.838831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd526-995f-41c6-acc1-6a4321410c3d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.844847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 448/934] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:19.852988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.879244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.879925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.880199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.880466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c78ca59d6dbd409ea40d1a9991c8263c/roles/ce6853f1894b4b67aeb6e1982c2c4008` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.882613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=c78ca59d6dbd409ea40d1a9991c8263c, role_id=ce6853f1894b4b67aeb6e1982c2c4008)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.884101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093378b-9b51-4ac5-98a5-99739c78cc5b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.900238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 487/935] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:19 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/c78ca59d6dbd409ea40d1a9991c8263c/roles/ce6853f1894b4b67aeb6e1982c2c4008 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:19.905253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2eb7-54a6-4610-a3a3-3d635840655e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.906078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2eb7-54a6-4610-a3a3-3d635840655e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.907262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2eb7-54a6-4610-a3a3-3d635840655e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.914094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 488/936] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:19.916443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.916988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.917472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.949988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:19.952180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:19.979402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b2859-6f92-49b6-b753-a1c8724ddb25 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.022539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.023793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.024270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.024987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.026262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:20.026761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.028954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.037261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 449/937] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:20.043373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:20.047131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 489/938] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:20.074974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:20.157974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b546b7-f0f3-4836-8881-fcba2abc9665 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.194858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.195471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.195772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.196041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.196692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.198052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.223050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 450/939] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:20.331569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:20.369039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:20.442096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fef8d-8be2-48e0-946a-9855e4ba9177 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.487243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.488328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.488814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.489306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.490087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.491709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:20.501589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:20.502606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:20.503454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:20.506193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 490/940] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:20.512492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:20.609554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:20.633240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f0ee7-5a60-4d81-bc39-45c3f896a5bc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.670714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.671332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.671646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.671914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.672599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:20.672905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.674279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.679597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 451/941] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:20.688768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:20.792240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:20.801718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ebf7-2d7f-46f5-b66e-f79f0f614e39 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.837240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.838213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.838697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.839189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.841084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:20.841594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.843153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.848759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 491/942] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:20.854843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:20.967723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8b75-e6a7-4a86-b738-5d2d0a652601 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.007508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.008138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.008420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.008695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.009409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] RBAC: Adding query filter params (interface=public, service_id=e27116008e93423da3a436117911a5fb, region_id=RegionOne) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:21.009720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.011149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1b43d-563f-40e5-a7fc-73acd5602233 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.018334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 452/943] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:20 2022] GET /identity/v3/endpoints?service_id=e27116008e93423da3a436117911a5fb&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:21.024168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.129388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebf8e9-d561-4d42-aa35-10f2243bab44 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.129870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ac1a-e1af-4a16-8a41-d704ad5e32e6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.130158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ac1a-e1af-4a16-8a41-d704ad5e32e6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.130435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ac1a-e1af-4a16-8a41-d704ad5e32e6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.132166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 453/944] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:21.137142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.137488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.137770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.155838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:21.160162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:21.161351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:21.166827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.167450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.168426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.168938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.169575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.170947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.177615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 492/945] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:21 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:21.241457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 454/946] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:21.520685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.550277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:21.632450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea4fd-66e7-4497-acf0-0a244bce2f9d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.669663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.670348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.670635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.670908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.671629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.673074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.675714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING keystone.server.flask.common [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:36:21.695069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.sql.core [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:36:21.695069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:36:21.695069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [parameters: {'id': '171d6e481ba9457e92501a180eed8973', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:36:21.695069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87988) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-b2e09fa5-f977-4ca2-9b0d-f099cf877985 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 171d6e481ba9457e92501a180eed8973.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 171d6e481ba9457e92501a180eed8973. Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:36:21.712121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application result.read() Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 17:36:21.713010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application yield resource Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise self.value Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application session.commit() Aug 24 17:36:21.713994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:36:21.715464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:36:21.716434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application result.read() Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:36:21.717265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [parameters: {'id': '171d6e481ba9457e92501a180eed8973', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:21.718134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:21.720341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 171d6e481ba9457e92501a180eed8973. Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:21.722631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 493/947] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Aug 24 17:36:21 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:21.732081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.828634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac97e7-3915-42eb-b95e-3e2dffa9aa56 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.848004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb86b-a14f-440b-a814-c88d45927ee1 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.848645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb86b-a14f-440b-a814-c88d45927ee1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.848645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029bb86b-a14f-440b-a814-c88d45927ee1 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.857490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 494/948] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:21.858710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1922138-e414-4420-979d-1b3b03f015da None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.858988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1922138-e414-4420-979d-1b3b03f015da None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.859256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1922138-e414-4420-979d-1b3b03f015da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.873180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.874315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.874682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.875387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.879310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e1922138-e414-4420-979d-1b3b03f015da None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:21.880970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e1922138-e414-4420-979d-1b3b03f015da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:21.883361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:21.885706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.887493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:21.903298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:21.904506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:21.906374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 455/949] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:21.911103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.956924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 495/950] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:22.019972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc74e42-82cf-464d-becc-32f562930b63 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.054957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.055594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.055871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.056139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.056797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:22.057096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.058557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76c1bd-01b5-4827-adcc-c31417243225 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.075978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 456/951] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:22.230942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1922138-e414-4420-979d-1b3b03f015da None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.241805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:22.341878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1922138-e414-4420-979d-1b3b03f015da None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.372630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783afc-995a-4003-ad20-7693a5701680 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.372976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783afc-995a-4003-ad20-7693a5701680 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.373190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783afc-995a-4003-ad20-7693a5701680 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.375855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 457/952] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:22.379323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.379944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.380225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.380495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.380929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.381421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.381897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.393338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:22.398352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:22.399309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:22.400226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:22.401033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:22.401033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:22.407560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 496/953] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:22.409530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.465133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:22.489824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 458/954] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:22.496089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:22.525161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75e1f5-8c31-4439-9b8f-560ba89e2f16 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.560720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.561359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.561633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.561901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.562557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:22.562857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.564517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.574524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 497/955] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:22.580216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.688940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d06857-f4de-4647-8f38-500a2f051419 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.725318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.725956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.726233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.726502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.737654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.739116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:22.755111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:22.757676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:22.760024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:22.760024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:22.760024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:22.760024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 459/956] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:22.760564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.764076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.820920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:22.867057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe6228-957a-401d-b647-2fd6b1e49e45 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.896298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:22.903299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8ab4-6be0-4910-8df2-d4bdd50ba571 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.904651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.905575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.906109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.906584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.907350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.908940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:22.918500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:22.919601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:22.920487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:22.923056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 498/957] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:22.933119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.941809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.942429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.942705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.942974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.943662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:22.943965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.945771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.960889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 460/958] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:22.967453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.062777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e0f32-a4e2-4887-b579-473298e95651 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.093674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.099450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add768f-6fe1-463b-96cc-5abd42541a53 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.110290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.111106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.111559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.112431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.113242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:23.114117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.116220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.128704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 499/959] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:23.130537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.148444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.149172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.149451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.149852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.163174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.165234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-0b7cbc87-24e6-4db5-b7e3-813299173874 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:23.179238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:23.181552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:36:23.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:23.187393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 461/960] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:36:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:23.187393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c7ca-ba27-4869-a729-3d091d28c98d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.187393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c7ca-ba27-4869-a729-3d091d28c98d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.187875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649c7ca-ba27-4869-a729-3d091d28c98d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.191529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 462/961] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:23.191529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649c7ca-ba27-4869-a729-3d091d28c98d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.207142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.255153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0247-dc43-43ee-b94c-5aaaaf226ca2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.291964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.293098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.293390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.293655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.294291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:23.294592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.295997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8273413-16c5-4662-8a36-d5d7ff631b29 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.301688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 500/962] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:23.310069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649c7ca-ba27-4869-a729-3d091d28c98d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.310623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.311161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.311724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.344012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:23.345588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:23.357183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.358146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.358604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.359050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.360276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:23.360808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.362540 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.389681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 463/963] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:23.391004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.415686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.435278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 501/964] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:23.518620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd796ad4-f258-496b-8bc9-70f6104f3d47 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.556350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.556965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.557247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.557520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.633681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.635611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.654228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.736840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.743375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 464/965] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:23.750237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.797583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.859041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43686a-58b4-4f15-9fc4-74cadd80e7a9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.861848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0df12b-cd61-4731-8923-994f8dbb4a3e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.879751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.903432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.904020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.904297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.904567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.905277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.905952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.906340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.906706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.907385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.908854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.916092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.917641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:23.918047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:23.919250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:23.920349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:23.926551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 502/966] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:23.932476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:23.934393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:23.935346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:23.936278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:23.936278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:23.936278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:23.936278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 465/967] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:23.943567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.064771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb485da-1cf0-47f3-8ec8-094fa2ef6802 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.066853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cf54f-b27c-4321-bacc-9a1e10e5dcdf None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.090496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.100939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.101717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.102057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.102403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.103098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:24.103466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.105873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aea6ef-d31f-4181-b753-31f54cbc66d9 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.115311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.116706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.117547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.117823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.118456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:24.118760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.119201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 466/968] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:24.120293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574615-6ef1-4f19-bb2e-ceee550c6af0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.120404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00557-fe16-4254-8e93-26797d5f33dd None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.120595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00557-fe16-4254-8e93-26797d5f33dd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.123510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b00557-fe16-4254-8e93-26797d5f33dd None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.123510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 467/969] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:24.125377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b00557-fe16-4254-8e93-26797d5f33dd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.126000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 503/970] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:24.129891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.130176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.130446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.148135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:24.149382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:24.160099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.223639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 504/971] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:24.223639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.243362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b00557-fe16-4254-8e93-26797d5f33dd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.279539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.280402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.280885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.281292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.288335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.290308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:24.302368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:24.303248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:24.304177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.306669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 468/972] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:24.316412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.353058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54ee22-db10-4e5f-aca0-abd24335fc7d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.368273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.398217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.399198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.399711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.400193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.401018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:24.401509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.403104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.408716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 505/973] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:24.414261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.435997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.448445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb97d8-90cb-49f0-be0a-d8504cb32986 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.484711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.485635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.485853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.486124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.486859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:24.487122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.488792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.503179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 469/974] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:24.504080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.530725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061188c-5d39-4294-bf65-0d98ba68470b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.567601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.568208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.568482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.568750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.569396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.570753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.591258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 506/975] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:24.592111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.628614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6d2e9-060f-41d1-bb5b-b1e74a53723d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.631841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.680716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.681389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.681772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.682140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.682829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.684476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.687301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING keystone.server.flask.common [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:36:24.702769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.sql.core [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:36:24.702769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:36:24.702769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [parameters: {'user_id': 'f4ab0860962e4543a414cb207d15f432', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:36:24.702769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87987) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result.read() Aug 24 17:36:24.709883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:36:24.710863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application yield resource Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise self.value Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application session.commit() Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:36:24.711926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:36:24.712830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:36:24.715248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application result.read() Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:36:24.716365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f4ab0860962e4543a414cb207d15f432', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:24.717817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 17:36:24.718819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.720090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 470/976] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:24.721779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.731905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.755078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269ef4e-5a54-47ef-af58-6a3dbfcfa121 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.800525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.801408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.801684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.801952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.802673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] RBAC: Adding query filter params (role.id=ce6853f1894b4b67aeb6e1982c2c4008, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=c78ca59d6dbd409ea40d1a9991c8263c) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:24.803021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.804537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.812845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 507/977] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/role_assignments?user.id=c78ca59d6dbd409ea40d1a9991c8263c&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=ce6853f1894b4b67aeb6e1982c2c4008 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:24.851191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60262c25-6414-41ed-8c38-e45b5bd0be2b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.892593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.893256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.893535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.893810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.903616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.905320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b718a4c0-1ecb-42d7-a6fd-62cad0dd769d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:24.918072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:24.919085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:24.920077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 471/978] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:24.926979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:25.032358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:25.039683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806e722-1a5d-4fa8-b707-02a947a34cad None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:25.076042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:25.076647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.076919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.077189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.077876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:25.078171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:25.079606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e8f54-de3d-483c-90c7-c8c4efd066ce None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:25.117338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 508/979] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:25.354234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b6b5-9534-4e54-babf-dc18e851ff25 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.354543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b6b5-9534-4e54-babf-dc18e851ff25 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.354759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b6b5-9534-4e54-babf-dc18e851ff25 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.359768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 472/980] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:25.365568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f918268-e0e6-4a17-a876-3b736b349c73 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.365861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f918268-e0e6-4a17-a876-3b736b349c73 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.366131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f918268-e0e6-4a17-a876-3b736b349c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.384586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2f918268-e0e6-4a17-a876-3b736b349c73 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:25.385785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2f918268-e0e6-4a17-a876-3b736b349c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:25.453439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 509/981] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:25.723622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27b6b5-9534-4e54-babf-dc18e851ff25 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:25.792695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:25.830316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27b6b5-9534-4e54-babf-dc18e851ff25 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:25.864400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:25.865000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.865277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.865549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.866201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:25.867603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78effa-78ca-46b4-ad6e-ad8dfad94eec None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:25.882331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 473/982] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:25 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:26.582173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385829c-28a5-4078-a64e-a7536707224e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:26.582701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385829c-28a5-4078-a64e-a7536707224e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:26.582701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385829c-28a5-4078-a64e-a7536707224e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:26.584251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 510/983] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:26.589852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1747-51cf-4bdb-ba38-697cb5fe9b1c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:26.590142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1747-51cf-4bdb-ba38-697cb5fe9b1c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:26.590411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd1747-51cf-4bdb-ba38-697cb5fe9b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:26.609366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-eddd1747-51cf-4bdb-ba38-697cb5fe9b1c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:26.611128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-eddd1747-51cf-4bdb-ba38-697cb5fe9b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:26.687835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 474/984] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:26.837847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac988-0754-4566-891f-d8dc569e9e7c None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:26.838142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac988-0754-4566-891f-d8dc569e9e7c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:26.838307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cac988-0754-4566-891f-d8dc569e9e7c None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:26.840565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 511/985] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:26.845772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:26.846061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:26.846341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:26.864773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:26.865959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:26.914060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:26.935019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 475/986] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:26.956396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cac988-0754-4566-891f-d8dc569e9e7c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.046895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.067591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cac988-0754-4566-891f-d8dc569e9e7c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.102717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.103267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.103573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.103845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.104413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.106375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:27.115242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:27.116225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:27.117052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.119812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 512/987] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:27.122185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.205080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.228456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9da3e-d81b-4de6-93d8-66f217dae6fd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.265544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.266165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.266443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.266713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.267339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:27.267673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.269054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.274291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 476/988] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:27.280768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.320560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288f80d-94b7-4789-874d-ba64f96e8d3f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.328516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.358309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.358852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.359125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.359390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] PATH_INFO: `/v3/services/image` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.360664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.362390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:27.372533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:27.373556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:27.376622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 513/989] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:27.379764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.388764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5547d-aff4-4c6e-acdb-e6de3561655c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.393603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.428549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.429154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.429430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.429690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.430303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:27.430596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.431983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.442195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 477/990] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:27.444004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.495202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098216c-1d86-44cc-87a2-af40b4b56399 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.532245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.533230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.533516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.533790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.534747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] RBAC: Adding query filter params (name=image) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:27.535060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.536472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.541679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 514/991] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:27.547575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.557336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31177abe-54b9-4f4b-81ae-903b2e8ace2e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.569578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.596457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.597069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.597350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.597647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.598317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] RBAC: Adding query filter params (interface=admin, service_id=e27116008e93423da3a436117911a5fb, region_id=RegionOne) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:27.598633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.600038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.605916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 478/992] 10.222.0.27 () {56 vars in 1249 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/endpoints?service_id=e27116008e93423da3a436117911a5fb&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:27.611698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:27.631256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.657968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc7fd0-13ed-4431-b451-044a7fbcdd92 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.695341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.696099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.696380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.696774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.697458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] RBAC: Adding query filter params (type=image) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:27.697757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.699865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ea5fd-eb72-44e2-b825-7f648136daa3 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.715637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 515/993] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:27.727220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ea41-8ec0-4219-9c5a-4ca31f14dd2d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:27.766738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:27.767363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.767685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.767955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.768570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:27.770023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e77096-4688-4fc8-a8d6-93c0b0bdd3ef None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:27.783582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 479/994] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:27 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:36:27.850240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89c28-2f34-4b2a-8ff6-f5d3aa3d6e74 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.850526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89c28-2f34-4b2a-8ff6-f5d3aa3d6e74 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.850802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89c28-2f34-4b2a-8ff6-f5d3aa3d6e74 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.852218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 516/995] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:27.858046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:27.858354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:27.858605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:27.879744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:27.880902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:27.915797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:27.958894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 480/996] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:28.231077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce89c28-2f34-4b2a-8ff6-f5d3aa3d6e74 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:28.336560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce89c28-2f34-4b2a-8ff6-f5d3aa3d6e74 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:28.371464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:28.372180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:28.372454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:28.372721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] PATH_INFO: `/v3/services/compute` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:28.373287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:28.374644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:28.383332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:28.384332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:28.385210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 517/997] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:28.391388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:28.493549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84825536-c196-4a74-bd8e-7c8c4e5867cd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:28.539087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:28.539740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:28.540023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:28.540296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:28.540922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:28.541224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:28.543130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:28.548227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 481/998] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:28.553816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:28.557290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:28.656289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69335d-937d-43cb-8521-538c2a18206c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:28.691055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:28.691696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:28.691979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:28.692251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:28.692872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] RBAC: Adding query filter params (type=compute) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:28.693175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:28.694532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:28.704845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 518/999] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:28.705782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:28.736896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:28.811592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f1d85-5867-4590-80f6-d62324e8aa29 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:28.845515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:28.846124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:28.846404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:28.846675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:28.847368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] RBAC: Adding query filter params (interface=public, service_id=4c1ea545ac5e45efb7e9951ea9fb4ac3, region_id=RegionOne) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:28.847705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:28.849072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513453e-e1be-4dd5-80ce-5190af95d22c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:28.858768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 482/1000] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:28 2022] GET /identity/v3/endpoints?service_id=4c1ea545ac5e45efb7e9951ea9fb4ac3&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:28.860162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:28.928459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:28.963354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec2936-871f-4cfd-a554-a6bc9b237b1e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:28.999576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:29.000180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:29.000462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:29.000733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:29.001331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:29.002691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a265e-1334-4be0-97ce-8ee79d7d950c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:29.008983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 519/1001] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:28 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:29.615643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d61f-f9bc-47f6-901a-695af29c6106 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:29.616190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d61f-f9bc-47f6-901a-695af29c6106 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:29.616190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19d61f-f9bc-47f6-901a-695af29c6106 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:29.617565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 483/1002] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:29.623203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484363a-caa6-4803-b825-ca46ac52e3b1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:29.623522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484363a-caa6-4803-b825-ca46ac52e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:29.623782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484363a-caa6-4803-b825-ca46ac52e3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:29.642468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4484363a-caa6-4803-b825-ca46ac52e3b1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:29.643740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4484363a-caa6-4803-b825-ca46ac52e3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:29.710165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3f42-f201-4eaf-8cdf-7a3b84f1732e None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:29.710470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3f42-f201-4eaf-8cdf-7a3b84f1732e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:29.710719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3f42-f201-4eaf-8cdf-7a3b84f1732e None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:29.711237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 520/1003] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:29.712206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 484/1004] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:29.717169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:29.717466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:29.717745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:29.735865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:29.737060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:29.791203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:29.803807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 521/1005] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:29.976868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca3f42-f201-4eaf-8cdf-7a3b84f1732e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:30.071646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:30.079819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:30.089804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca3f42-f201-4eaf-8cdf-7a3b84f1732e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:30.127405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:30.128050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.128331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.128603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.129248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:30.130624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:30.155574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 485/1006] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:29 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:30.190846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ab24-72b5-45a8-ad83-51153b2beab1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:30.241168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:30.241825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.242106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.242376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.242962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:30.244363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:30.253741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:30.254883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:36:30.256220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:30.259867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 522/1007] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Aug 24 17:36:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:30.261287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:30.370122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ec455-e754-44c3-841b-5c93dd152328 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:30.405283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:30.405895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.406173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.406447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.407073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:30.407375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:30.408788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:30.414469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 486/1008] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:30.419889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:30.472162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:30.534274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32299a7f-c9da-41fd-92f8-6df28daab08f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:30.570655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:30.571262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.571561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.571830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.572447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:30.572741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:30.574669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e3641-714f-4cca-8988-d86ee32cf31f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:30.584486 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 523/1009] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Aug 24 17:36:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:30.585775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:30.610398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:30.694713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b636a8-5d3d-4f6d-a1ea-dc934acc5aee None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:30.728567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:30.729168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.729444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.729710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.730883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:30.732271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8de9a-cbd4-4652-9b80-887e0362049a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:30.753791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 487/1010] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:30.940422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93d8c4-abf2-4878-afea-b4a7dc142f5c None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.940929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93d8c4-abf2-4878-afea-b4a7dc142f5c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.941007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93d8c4-abf2-4878-afea-b4a7dc142f5c None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.942453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 524/1011] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:30.947739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:30.947965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:30.948233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:30.967424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:30.968944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:31.045261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 488/1012] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:30 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:31.313018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93d8c4-abf2-4878-afea-b4a7dc142f5c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:31.415744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be93d8c4-abf2-4878-afea-b4a7dc142f5c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:31.429689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:31.451369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:31.452034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:31.452315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:31.452588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] PATH_INFO: `/v3/services/compute` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:31.453162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:31.454515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:31.463232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:31.464379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:36:31.465230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:31.467593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 525/1013] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:31.470623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:31.478958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:31.583011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8727df4-5f06-44f9-8add-c3639f2edc07 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:31.618511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:31.619122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:31.619397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:31.619693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:31.620832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] RBAC: Adding query filter params (name=compute) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:31.621139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:31.622517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:31.632101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 489/1014] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:31.633634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:31.736627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281989d-0beb-4142-8a93-2c43e2b8f62e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:31.772958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:31.773562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:31.773838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:31.774105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:31.774725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:31.775023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:31.776440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095bd70-e513-4d7b-bbb1-630d12110a11 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:31.786039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 526/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Aug 24 17:36:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:31.787566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:31.859697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:31.893580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499012a-9269-4690-bdfe-7dc08c571e7a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:31.926524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:31.927132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:31.927405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:31.927701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:31.928362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:31.929715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3605cb-877d-4585-8177-c0f6c3353160 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:31.951727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 490/1016] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:32.071325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2709a-50b5-4f90-9eba-8155f9c24c28 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.071980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2709a-50b5-4f90-9eba-8155f9c24c28 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.071980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2709a-50b5-4f90-9eba-8155f9c24c28 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.073903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 527/1017] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:32.079337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.079656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.079932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.100593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:32.101800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:32.180244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 491/1018] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:32.452082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2709a-50b5-4f90-9eba-8155f9c24c28 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:32.501393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:32.570155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2709a-50b5-4f90-9eba-8155f9c24c28 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:32.606199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153c58-e082-4243-9451-561d97c4054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:32.606904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c58-e082-4243-9451-561d97c4054b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.607184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c58-e082-4243-9451-561d97c4054b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.607451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c58-e082-4243-9451-561d97c4054b None admin] PATH_INFO: `/v3/services/image` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.608059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153c58-e082-4243-9451-561d97c4054b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:32.609436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153c58-e082-4243-9451-561d97c4054b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-08153c58-e082-4243-9451-561d97c4054b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:32.618458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:32.619423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:32.621376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:32.622435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 528/1019] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 17:36:32 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:32.626050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:32.680336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:32.709115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153670a-2f19-4d14-8253-1bb9a6ce25aa None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.709309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153670a-2f19-4d14-8253-1bb9a6ce25aa None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.709592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153670a-2f19-4d14-8253-1bb9a6ce25aa None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.711072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 529/1020] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:32.716058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.716858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.717337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.732276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f4923-f080-40d6-acd8-1173ceaf675b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:32.740462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:32.741889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:32.766789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:32.767386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.767686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.767953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.768591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] RBAC: Adding query filter params (name=image) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:32.768885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:32.770266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:32.779962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 492/1021] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:32.780945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:32.823147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 530/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:32.889959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f415dc-e98a-451a-b3a1-29d975decb76 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:32.907730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:32.926496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:32.927110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:32.927389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:32.927686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:32.928310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] RBAC: Adding query filter params (type=image) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:32.928615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:32.930009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:32.940316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 493/1023] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:32.941358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:32.979104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:33.046075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d366642-2130-4358-95b2-2b4ec9ad4827 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:33.079892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:33.080529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.080771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.081036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.081725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] RBAC: Adding query filter params (interface=public, service_id=5b3db6dfe66b4af29dd266dd311d0cb0, region_id=RegionOne) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:33.082029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:33.083398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:33.091701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 531/1024] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:36:32 2022] GET /identity/v3/endpoints?service_id=5b3db6dfe66b4af29dd266dd311d0cb0&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:33.095015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:33.095163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:33.200458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:33.203651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db0445-a0eb-44bc-ad8e-595bd97a881c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:33.204198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421a14-83f1-41e5-8691-cfe1c309bb5e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:33.241165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:33.241350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:33.241703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.241874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.242008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.242138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.242278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.242359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.242822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:33.244228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71157b-af96-4eed-80bc-9c94f76b15d6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:33.244739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:33.246680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:33.251507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 532/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:33.251787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 494/1026] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:36:33 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:33.258413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:33.365247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b282c-e10d-477b-92c3-91e028cd4edc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:33.401246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:33.402445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.402716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.402977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.403675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:33.405120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040227ef-156d-4f9a-9cc6-76c4594275e7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:33.435417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 533/1027] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:36:33 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:33.949874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a21e-d473-4c90-9b8b-3c4ad432fb1e None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.950369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a21e-d473-4c90-9b8b-3c4ad432fb1e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.950369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22a21e-d473-4c90-9b8b-3c4ad432fb1e None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.951852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 495/1028] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:33.956971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:33.957390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:33.957736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:33.976918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:33.981564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:34.002491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:34.059516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 534/1029] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:34.324568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22a21e-d473-4c90-9b8b-3c4ad432fb1e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:34.375653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:34.432357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22a21e-d473-4c90-9b8b-3c4ad432fb1e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:34.466502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:34.467134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:34.467413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:34.467713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:34.477868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:34.483075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:34.484190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:34.485321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:34.487811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 496/1030] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:34.490344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:34.603240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa21b19-5dc2-48a5-8989-5ec183cfeb7e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:34.638196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:34.638801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:34.639080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:34.639351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:34.640036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:34.640336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:34.641706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:34.654654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 535/1031] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:34.656169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:34.758912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8b679-bb3a-479d-b55f-66e8fb80b82b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:34.761392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:34.796597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:34.797319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:34.797601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:34.797872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:34.808794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:34.810233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:34.825959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:34.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:34.827878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:34.827878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:34.827878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:34.827878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 497/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:34.834404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:34.850300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:34.939912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021bd3-d4d7-4cb4-8bb4-d0bb9e99e06b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:34.974390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:34.975002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:34.975284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:34.975585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:34.976227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:34.976531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:34.977896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:34.990762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 536/1033] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:34.996946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.100138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4256f15-0c56-4ead-a095-63fd50b8e9f8 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.134665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.135353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.135661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.136499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.144498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.145940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-07a78b52-a067-49bf-87b8-4f53ab73ba05 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:35.157586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:35.158572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:35.159604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 498/1034] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:36:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:35.166138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.185392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d7db-1faa-47fd-a3f5-7e3b9355fcf7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.185900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d7db-1faa-47fd-a3f5-7e3b9355fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.186397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d7db-1faa-47fd-a3f5-7e3b9355fcf7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.188973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 499/1035] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:35.194260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.194486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.194768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.214801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:35.216038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:35.228734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.247260 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.280098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1594a7-4d06-4e0c-804c-af5b00ec9b68 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.293417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 500/1036] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:35.315283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.315916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.316192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.316461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.317176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:35.317475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.318953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b55a-bf58-4702-8b8f-eba38eee4524 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.341828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 537/1037] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:35.343273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.366849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7621-2cae-4117-be1c-fb72c34a1c93 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.367140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7621-2cae-4117-be1c-fb72c34a1c93 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.367437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7621-2cae-4117-be1c-fb72c34a1c93 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.368922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 538/1038] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:35.374351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.374664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.374941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.394135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:35.395332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:35.445971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ba1e-1178-46da-b8ae-9bf2b24eb5db None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.463343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 539/1039] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:35.481194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.481795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.482068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.482333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.497500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.556669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.558161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.562586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.624941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.667694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30739efb-127d-4ebc-ab23-49e3c243713b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.671909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 501/1040] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:36:35.673151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.703342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.704108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.704319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.704580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] PATH_INFO: `/v3/services/image` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.705152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.706496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:35.715228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:35.716257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:35.720531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 540/1041] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:35.722528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.725264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.779610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a97f2c-124c-4b04-bcd0-acca06a6b1fc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.815038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.815700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.815973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.816244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.822772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53b5ef-3ff7-4b03-833c-73d579dfce6d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.826028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.827504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:35.841691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:35.842857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:35.843796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:35.843796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:35.843796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:35.843796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 502/1042] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:35.843796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.850483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.862113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.862725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.863002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.863273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.863916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] RBAC: Adding query filter params (name=image) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:35.864212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.865576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.870999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 541/1043] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:35.871795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:35.943608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.948330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018ea6-c249-4fee-a5a0-02293267c15e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.969700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf138-bd82-427b-94e4-f3154680c535 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:35.982969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:35.983680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:35.983984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:35.984253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:35.985611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:35.987042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:35.988650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 503/1044] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:35.989408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.007331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.007973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.008282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.008529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.009509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:36.009819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.011877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.026101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 542/1045] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:36.027572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.078099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:36.097844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12aab-6db1-4058-847c-0b30ba38b672 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.129985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7362c1-7750-467a-ac41-ceda8b74779a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.132051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86150-4f61-44de-a313-431964177c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.132664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86150-4f61-44de-a313-431964177c56 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.132943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86150-4f61-44de-a313-431964177c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.133211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86150-4f61-44de-a313-431964177c56 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.133836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86150-4f61-44de-a313-431964177c56 None admin] RBAC: Adding query filter params (type=image) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:36.134137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86150-4f61-44de-a313-431964177c56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.135522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86150-4f61-44de-a313-431964177c56 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.140852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 504/1046] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:36.141631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86150-4f61-44de-a313-431964177c56 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.168798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.169417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.169704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.169975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.170613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.171082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:36.172087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.193833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 543/1047] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:36:36 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:36.194632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.255138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86150-4f61-44de-a313-431964177c56 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.296176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.296836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.297121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.297394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.305781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.307265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.310253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9e55-f43e-4887-ba37-c94cc4281791 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.312800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:36.320461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:36.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:36.322388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:36.327364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 505/1048] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:36.328586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.356581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.356581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.356887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.357129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.357777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.359208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.381885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 544/1049] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:36.443988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:36.455473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208ace4-2439-4bfc-a67c-7f697ef9ac4d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.500093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.500710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.500988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.501255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.501951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:36.502257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.503757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b35995-0b3e-4f9a-a5fb-7b1affda4ea9 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.523081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 506/1050] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:36.525698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:36.631324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65915a28-869d-4bbc-afb9-1b9b08438122 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:36.664585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:36.665199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:36.665460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:36.665768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:36.666439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] RBAC: Adding query filter params (role.id=ce6853f1894b4b67aeb6e1982c2c4008, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:36.666742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:36.668782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4ac32-eb93-49cc-b5f3-1ad5de04f062 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:36.674363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 545/1051] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:36 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=ce6853f1894b4b67aeb6e1982c2c4008 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:38.212923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ad61-c39e-46ff-8be5-e3dc54f075cf None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.213454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ad61-c39e-46ff-8be5-e3dc54f075cf None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.213454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885ad61-c39e-46ff-8be5-e3dc54f075cf None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.215439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 507/1052] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:38.220636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d08867-992e-439f-be7d-28cc62ab9410 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.221047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d08867-992e-439f-be7d-28cc62ab9410 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.221208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d08867-992e-439f-be7d-28cc62ab9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.239831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-09d08867-992e-439f-be7d-28cc62ab9410 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:38.241044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-09d08867-992e-439f-be7d-28cc62ab9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:38.258449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:38.311735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 546/1053] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:38.368789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bbb7-f2bd-4b16-bf24-d704f5265bb4 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.369085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bbb7-f2bd-4b16-bf24-d704f5265bb4 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.369350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16bbb7-f2bd-4b16-bf24-d704f5265bb4 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.371369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 508/1054] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:38.376519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618feb85-6791-4d4f-b171-aba7284f2042 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.376811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618feb85-6791-4d4f-b171-aba7284f2042 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.377088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618feb85-6791-4d4f-b171-aba7284f2042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.395601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-618feb85-6791-4d4f-b171-aba7284f2042 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:38.396798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-618feb85-6791-4d4f-b171-aba7284f2042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:38.464490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 547/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:38.546195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289b03-aba4-4ce3-aab0-45cf8000a4b7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.546468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289b03-aba4-4ce3-aab0-45cf8000a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.546739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289b03-aba4-4ce3-aab0-45cf8000a4b7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.548250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 509/1056] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:38.553077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.553360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.553631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.574185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:38.575369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:38.579928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289b03-aba4-4ce3-aab0-45cf8000a4b7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:38.651936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 548/1057] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:38.691342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289b03-aba4-4ce3-aab0-45cf8000a4b7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:38.726428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:38.726987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.727261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.727598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.733039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:38.739014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:38.743726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:38.744686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:38.746160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 510/1058] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:38.750664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:38.751444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:38.761622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:38.843337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435377a6-1ae4-4c6c-b65c-2b19d48a7e87 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:38.867734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684afee-b904-4c3e-8115-aafdfead9f24 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:38.879923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:38.880565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.880817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.881092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.913607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:38.914532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.914958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.915445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.917946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:38.918414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:38.919962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:38.929385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 511/1059] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:38.930199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:38.953563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:38.955006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:38.970060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.001626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.034397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003fea9-6df7-4cc3-8d36-4ab783cbf876 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.053359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 549/1060] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:36:39.054193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.070302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.070941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.071217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.071499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.082303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:39.086933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:39.088067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 512/1061] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:39.089680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.154297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce3972-3e5b-42d4-818f-9593147c9a44 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.167764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.187134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dafd1-3a34-49da-ac7e-f3ae7c6405f0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.188777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.189396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.189669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.189934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] PATH_INFO: `/v3/users/e62e0dc15fa34d6e9613964806d4dd53` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.205363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] RBAC: Authorizing `identity:get_user(user_id=e62e0dc15fa34d6e9613964806d4dd53)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.206807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.208556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 550/1062] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/users/e62e0dc15fa34d6e9613964806d4dd53 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:39.209329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.216234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.224622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.225252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.225695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.225965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.237360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.238793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:39.252434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:39.253472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:39.254383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:39.254383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:39.254383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:39.254383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 513/1063] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:39.254383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.312173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce078ed-ba82-4b9d-97ae-ea1e3053f628 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.346092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.346697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.346979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.347252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.347946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:39.348243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.349608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.353342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2cb78-9613-4c7f-b2fd-e43e485b124e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.358020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 551/1064] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:39.359081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.390086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.390692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.390968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.391225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.393769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.440725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.459843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf0a79-382b-41a7-a512-32f45135e401 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.460956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.462409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.493847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.494441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.494724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.495066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.496231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:39.496532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.497885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.510109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 552/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:39.511334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.585815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 514/1066] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:39.592467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.628115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.636200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d81e-7e15-415f-b4ee-ec830969a82b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.678108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821823d-f980-43b3-b703-922441254e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.678890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821823d-f980-43b3-b703-922441254e00 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.679174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821823d-f980-43b3-b703-922441254e00 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.679454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821823d-f980-43b3-b703-922441254e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.686223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.692041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821823d-f980-43b3-b703-922441254e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.693504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821823d-f980-43b3-b703-922441254e00 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.708842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bfb42-ed8a-458d-b871-f547073b2eaa None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-5821823d-f980-43b3-b703-922441254e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:39.713081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:39.714615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:39.715568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:39.715568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:39.715568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:39.715568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 553/1067] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:39.720601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821823d-f980-43b3-b703-922441254e00 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.749781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.750438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.750714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.750987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.752123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.753597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.755263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 515/1068] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:39.762022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.828582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821823d-f980-43b3-b703-922441254e00 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.862539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9bbb4-8951-49ab-b99f-17951d538049 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:39.862938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.863564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.863839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.864210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.864725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:39.865027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.866375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.878524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 554/1069] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:39.883978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:39.892867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:39.897333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:39.897943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:39.898219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:39.898488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:39.899202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=345b78d36cf14c2b87b6d80166b160b4, user.id=e62e0dc15fa34d6e9613964806d4dd53) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:39.899580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:39.900974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:39.906504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 516/1070] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/role_assignments?user.id=e62e0dc15fa34d6e9613964806d4dd53&scope.project.id=345b78d36cf14c2b87b6d80166b160b4&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:39.992209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea081cbc-19d0-46f1-9159-7b6adb5de194 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.027520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.028078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.028349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.028613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.036266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.037720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:40.050125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:40.051293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:40.052571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 555/1071] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:36:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:40.057268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.073865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:40.159053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c153496-2d4c-46c7-9e17-df77813178c1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.193224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.193828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.194106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.194376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.195083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:40.195385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.196835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.214874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 517/1072] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:40.220737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.287722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:40.323406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8935-ab94-4f5e-8c44-7a28e87dfd96 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.356576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.357200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.357478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.357748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.368182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.369617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:40.383722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:40.384843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:40.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:40.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:40.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:40.386534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 556/1073] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:40.392859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.460357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:40.490754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c6a20-9b56-4519-88b1-719424598dec None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.529379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.529991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.530264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.530529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.531159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:40.531455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.532836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.544128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 518/1074] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:40.549705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.648212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2a2d6-2b95-4de3-87d4-7f4b3eeed7f0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.664404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:40.680746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.681376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.681665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.681937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.688979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.690414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:40.702654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:40.703722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:40.704671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:40.707741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 557/1075] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:40.709532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.809963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b625b-c458-482f-878c-363b59bab8c3 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.837327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:40.843952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224365-3980-4834-a1b1-2694167df76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.844556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224365-3980-4834-a1b1-2694167df76e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.844827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224365-3980-4834-a1b1-2694167df76e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.845091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224365-3980-4834-a1b1-2694167df76e None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.845764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224365-3980-4834-a1b1-2694167df76e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:40.846060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224365-3980-4834-a1b1-2694167df76e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.847487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224365-3980-4834-a1b1-2694167df76e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.860135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 519/1076] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:40.866045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.964222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94818e-2ea9-49e9-8c4a-044bd83e79de None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.999208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.999976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:41.000254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:41.000521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/ce6853f1894b4b67aeb6e1982c2c4008` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:41.002656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=1a6e9e58daeb47abb6277e7c2bf6bdcb, role_id=ce6853f1894b4b67aeb6e1982c2c4008)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:41.004125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23525a-5132-45f8-ae1b-dfad11b8a1af None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:41.018407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 558/1077] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:40 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/1a6e9e58daeb47abb6277e7c2bf6bdcb/roles/ce6853f1894b4b67aeb6e1982c2c4008 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:41.802882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c142d1-9648-4d16-98ea-583aa81129e9 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:41.803384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c142d1-9648-4d16-98ea-583aa81129e9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:41.803384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c142d1-9648-4d16-98ea-583aa81129e9 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:41.804732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 520/1078] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:41.809913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:41.810353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:41.810449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:41.828681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:41.829866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:41.858710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:41.897011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 559/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:42.165379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c142d1-9648-4d16-98ea-583aa81129e9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.265622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c142d1-9648-4d16-98ea-583aa81129e9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.298753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.299368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.299670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.299937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.309649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:42.314264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:42.315375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:42.316397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 521/1080] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:42.321513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.419141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a24c1-6fb1-40b5-8876-70ef8dbe1289 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.452601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.453201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.453478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.453749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.454400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:42.454700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:42.456090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:42.464733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 560/1081] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:42.470571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.521526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:42.571243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11083f8-e143-4d34-a529-a4bad4de9a12 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.607969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.608602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.608882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.609151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] PATH_INFO: `/v3/users/e62e0dc15fa34d6e9613964806d4dd53` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.610538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] RBAC: Authorizing `identity:get_user(user_id=e62e0dc15fa34d6e9613964806d4dd53)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:42.612017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:42.613624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 522/1082] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:42 2022] GET /identity/v3/users/e62e0dc15fa34d6e9613964806d4dd53 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:42.619364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.650338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:42.718483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76989959-c54d-4c0e-b90c-cbd7965216cd None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.752371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.753004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.753285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.753567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.754686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:42.756151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac30a52-cdd0-4513-9456-5adf350d4b42 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:42.757795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 561/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:42 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:42.763874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.861901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b96f0-3e4a-457b-bf4b-6a5e2f54136f None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.879313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:42.896913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.897572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.897851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.898120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4/users/e62e0dc15fa34d6e9613964806d4dd53/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.900063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] RBAC: Authorizing `identity:create_grant(project_id=345b78d36cf14c2b87b6d80166b160b4, user_id=e62e0dc15fa34d6e9613964806d4dd53, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:42.902057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:42.905195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b3fd1-1589-4ec0-a568-f8489a8713df None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.905650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b3fd1-1589-4ec0-a568-f8489a8713df None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.905739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b3fd1-1589-4ec0-a568-f8489a8713df None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.907185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 562/1084] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:42.912561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.912846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.913124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.916073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 523/1085] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:42 2022] PUT /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4/users/e62e0dc15fa34d6e9613964806d4dd53/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:42.935714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:42.936965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.010146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 563/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:43.278556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.377352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac90e-db9e-45dd-8b7a-af4c9d97ac7b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.410807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.411437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.411747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.412021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.422360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:43.427039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:43.428053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:43.429082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 524/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:43.434882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.461809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:43.533081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292c60d-7513-49aa-bc2b-058621271a05 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.566585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.567178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.567448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.567740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.568385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:43.568679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.570026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.578288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 564/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:43.584026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.623771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:43.683070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd147139-2eac-4323-a346-2ddc08886f9e None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.716080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.716703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.716981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.717249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.727094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.728572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:43.742771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:43.743910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:43.745564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:43.745564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:43.745564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:43.745564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 525/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:43.752015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.839062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:43.848747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d35f-a468-49a6-a61a-aab5abc7e734 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.881476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.882074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.882346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.882608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.883230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:43.883551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.884902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.896258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 565/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:43.902453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.993384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.003019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba3937-6995-4ed7-8b97-f9e83a366090 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.036269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.036888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.037161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.037425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.044903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.046327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:36:44.056963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:44.057953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:44.058936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 526/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:44.064001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:44.159981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabac8b-76c2-4015-9cc7-80a1712484f7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.192786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.193377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.193647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.193910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.194611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:44.194906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.196343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.213231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 566/1092] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Aug 24 17:36:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:44.219571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:44.316668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351e24-bd76-4904-8b64-cd358ad3304a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.349973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.350577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.350854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.351122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.369373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.418345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.419739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.516790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 527/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:36:44.521907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:44.532579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.621925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb050153-59e2-4c73-a5da-34e8a694a8ee None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.656182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.656813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.657090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.657357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.667651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.669138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-f308e644-1ea4-4093-9bbb-fa0c28bc038c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:36:44.683130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:44.684259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:36:44.685677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:36:44.685677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:36:44.685677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:36:44.685677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 567/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:44.689953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:44.739313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.784500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3c9d-7131-489d-9185-038673958f95 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.784776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3c9d-7131-489d-9185-038673958f95 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.785050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3c9d-7131-489d-9185-038673958f95 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.786375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 568/1095] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:36:44.789608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fd03-db1b-49a3-91ec-a3edddaac5eb None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.791148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.791462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.791843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.809084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:44.810274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:44.824622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.825234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.825514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.825785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.826416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:44.826714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.828121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.840034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 528/1096] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:44.845601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:44.870919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.883539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 569/1097] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:36:44.942324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d89cb-1bda-4538-b17b-d696c29852c0 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:44.952694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:44.976866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:44.977486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:44.977760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:44.978024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:44.985269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:44.986689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:36:44.998162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:44.999221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:45.000560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 529/1098] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:36:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:36:45.005157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.101671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb50b93-c6ea-41fc-9e44-90af693f4b3b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.134514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.135112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.135445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.135684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.136363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:45.136662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:45.138079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:45.149158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 570/1099] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:45.154968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.155313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.204781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:45.254656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31478a94-4f99-4474-8df6-4f7dfbeac1b9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.256294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f16f7-d5a9-4e4c-b6cc-736ee9394e8b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.287339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.287963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.288235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.288498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.288970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.289201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] RBAC: Adding query filter params (role.id=ce6853f1894b4b67aeb6e1982c2c4008, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:45.289498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:45.289637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.289921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.290190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.290910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:45.298917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 571/1100] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/role_assignments?user.id=1a6e9e58daeb47abb6277e7c2bf6bdcb&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=ce6853f1894b4b67aeb6e1982c2c4008 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:45.301555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:36:45.306292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:36:45.307880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:36:45.308751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:36:45.311680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 530/1101] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:36:45.314363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.419974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268385-cf25-4a25-a7b1-7224a050a06e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.434810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:45.457551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.458076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.458349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.458612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.459258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:45.459592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:45.460944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:45.469862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 572/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:45.475905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.508359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:45.582195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfce91-2a69-4b29-8e4e-0154dfe47b1c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.616534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.617169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.617451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.617726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] PATH_INFO: `/v3/users/e62e0dc15fa34d6e9613964806d4dd53` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.618860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] RBAC: Authorizing `identity:get_user(user_id=e62e0dc15fa34d6e9613964806d4dd53)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:45.620922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:45.622680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 531/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/users/e62e0dc15fa34d6e9613964806d4dd53 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:45.628727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:45.731470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb38acb-db6a-448f-92fd-f9d9e5ed63b5 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:45.766658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:45.767264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:45.767571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:45.767844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:45.813510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:45.837958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:45.840009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:45.944795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 573/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:45.951669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:46.059709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:46.063564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4ae-e320-43c2-9dc9-7caedd973a29 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:46.100419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:46.101052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:46.101325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:46.101593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:46.102695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:46.104727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1c07-2820-476e-86f9-63f61dbe7973 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:46.106513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 532/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:45 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:46.112991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:46.197021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:46.219526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab82e-9c81-480f-bff1-632bb56a6c26 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:46.252467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:46.253053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:46.253331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:46.253596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:46.254287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=345b78d36cf14c2b87b6d80166b160b4, user.id=e62e0dc15fa34d6e9613964806d4dd53) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:46.254586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:46.256045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48633a61-6f4c-4ab3-8c77-a8de8f08295c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:46.261485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 574/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:46 2022] GET /identity/v3/role_assignments?user.id=e62e0dc15fa34d6e9613964806d4dd53&scope.project.id=345b78d36cf14c2b87b6d80166b160b4&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:48.143540 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a108-8424-475f-b1b1-e18a1ce14697 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.144074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a108-8424-475f-b1b1-e18a1ce14697 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.144074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a108-8424-475f-b1b1-e18a1ce14697 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.145411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 533/1107] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:48.150536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.151437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.151672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.169846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:48.171044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:48.235812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 575/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:48.503276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8a108-8424-475f-b1b1-e18a1ce14697 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:48.602415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8a108-8424-475f-b1b1-e18a1ce14697 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:48.635939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:48.636563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.636842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.637113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.638223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:48.639671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5b1b4-1d87-4c8e-935d-20243df8b4f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:48.641239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 534/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:48.647918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:48.715257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:48.745215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db64ca-f1c5-464f-9a62-98fe4bfbeaa6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:48.778413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:48.778949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:48.779228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:48.779518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:48.780171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:48.781598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e3463-54dc-418a-8d39-60377f79a0d9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:48.801800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 576/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:36:48 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:50.742593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df5cd-9191-453b-b39f-4f5d8aea4bf5 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.743141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df5cd-9191-453b-b39f-4f5d8aea4bf5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.743141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4df5cd-9191-453b-b39f-4f5d8aea4bf5 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.745154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 535/1111] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:50.750384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:50.750757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:50.750924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:50.769090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:50.770280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:50.837372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 577/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:51.108583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4df5cd-9191-453b-b39f-4f5d8aea4bf5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.143721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.208025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4df5cd-9191-453b-b39f-4f5d8aea4bf5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.240661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.241273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.241551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.241821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] PATH_INFO: `/v3/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.243196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] RBAC: Authorizing `identity:get_role(role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.245114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.246421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 536/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:51 2022] GET /identity/v3/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:51.252237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.305236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.349773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58123-307f-403c-a675-6ba84e33f569 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.382530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.383084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.383360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.383656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] PATH_INFO: `/v3/users/4ab519681e0b45818647b9c7690ca7c9` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.398075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] RBAC: Authorizing `identity:get_user(user_id=4ab519681e0b45818647b9c7690ca7c9)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.399529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.401659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 578/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:51 2022] GET /identity/v3/users/4ab519681e0b45818647b9c7690ca7c9 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:51.408629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.505963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4a858-765a-4f0b-8dd3-ecf711be1a80 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.507464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.541140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.542277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.542554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.542820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.607980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.609683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.705081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 537/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:51.710961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.804835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea890f-a52b-4677-891e-85ed11db3747 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.819322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.838936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.839574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.839851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.840119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.841207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.842620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d1ec-af01-4e9c-9b89-b288fb37fab4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.844238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 579/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:51 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:51.850231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.852425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.946880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514b4c4-3fb7-4ffa-b368-5fd8356b7c1d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.980971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.981562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.981837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.982102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.982811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=345b78d36cf14c2b87b6d80166b160b4, user.id=4ab519681e0b45818647b9c7690ca7c9) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:51.983113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.984838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9008e3-a907-44e0-a4bb-5ee3f45543b4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.990038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 538/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:51 2022] GET /identity/v3/role_assignments?user.id=4ab519681e0b45818647b9c7690ca7c9&scope.project.id=345b78d36cf14c2b87b6d80166b160b4&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:53.862022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b45b-4cee-4a40-b8f9-2e4cbef05258 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:53.862533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b45b-4cee-4a40-b8f9-2e4cbef05258 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:53.862533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b45b-4cee-4a40-b8f9-2e4cbef05258 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:53.863846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 580/1118] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:53.869126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:53.870183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:53.870333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:53.888453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:53.889638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:53.948239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:53.956197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 539/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:54.220578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b45b-4cee-4a40-b8f9-2e4cbef05258 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:54.317879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b45b-4cee-4a40-b8f9-2e4cbef05258 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:54.350113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:54.350719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:54.350990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:54.351252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] PATH_INFO: `/v3/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:54.352627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] RBAC: Authorizing `identity:get_role(role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:54.354536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:54.355843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 581/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:54 2022] GET /identity/v3/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:54.361775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:54.458855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e557af-c936-43ea-94f2-e54685c868ed None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:54.492109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:54.492671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:54.492954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:54.493223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] PATH_INFO: `/v3/users/4ab519681e0b45818647b9c7690ca7c9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:54.494873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] RBAC: Authorizing `identity:get_user(user_id=4ab519681e0b45818647b9c7690ca7c9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:54.496344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:54.497926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 540/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:54 2022] GET /identity/v3/users/4ab519681e0b45818647b9c7690ca7c9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:54.503642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:54.552459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:54.601673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f457b-17ba-4478-a412-a25dc5933775 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:54.635596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:54.636222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:54.636500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:54.636773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:54.637884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:54.639308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f48670-bb0a-43f4-8896-e2d8bd2f4031 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:54.640953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 582/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:54 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:54.646970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:54.708948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:54.746440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ac08-c1c1-4feb-8856-d87df5ae8949 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:54.778514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:54.779162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:54.779442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:54.779746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4/users/4ab519681e0b45818647b9c7690ca7c9/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:54.781880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] RBAC: Authorizing `identity:create_grant(project_id=345b78d36cf14c2b87b6d80166b160b4, user_id=4ab519681e0b45818647b9c7690ca7c9, role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:54.783309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f783aa3-9502-4e2d-929b-65a7e49a0804 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:54.796289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 541/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:36:54 2022] PUT /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4/users/4ab519681e0b45818647b9c7690ca7c9/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:56.655380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faefd0c-f2af-4358-ab97-60b3d7f73fc8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.655952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faefd0c-f2af-4358-ab97-60b3d7f73fc8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.655952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faefd0c-f2af-4358-ab97-60b3d7f73fc8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.657302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 583/1124] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:56.662654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.663071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.663231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.680875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:56.682564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:56.748137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 542/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:36:57.013700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faefd0c-f2af-4358-ab97-60b3d7f73fc8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.112073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faefd0c-f2af-4358-ab97-60b3d7f73fc8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.129049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.145538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.146155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.146440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.146714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] PATH_INFO: `/v3/roles/9245c00a908549cf9b22dd1e82d22bb2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.148126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] RBAC: Authorizing `identity:get_role(role_id=9245c00a908549cf9b22dd1e82d22bb2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.149485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.150806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 584/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/roles/9245c00a908549cf9b22dd1e82d22bb2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:57.156478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.210586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.254423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaced4f-0b58-4686-9ed7-00b82e2adc31 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.287759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.288347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.288623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.288893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] PATH_INFO: `/v3/users/4ab519681e0b45818647b9c7690ca7c9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.290008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] RBAC: Authorizing `identity:get_user(user_id=4ab519681e0b45818647b9c7690ca7c9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.291504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.293073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 543/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/users/4ab519681e0b45818647b9c7690ca7c9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:57.298917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.395393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fca70f-ae0f-4dbd-8c39-25f102dd96d0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.428486 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.429079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.429364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.429636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.484370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.497517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.498956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.593781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 585/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:36:57.599803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.696499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6818d40-1977-4cae-ae7f-81e9cfb803ac None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.719395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.731031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.731685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.731964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.732231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] PATH_INFO: `/v3/projects/345b78d36cf14c2b87b6d80166b160b4` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.733340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] RBAC: Authorizing `identity:get_project(project_id=345b78d36cf14c2b87b6d80166b160b4)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.735034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147169f-7934-49a2-829d-a81ae0ec96f0 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.736692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 544/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/projects/345b78d36cf14c2b87b6d80166b160b4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:57.742763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.824133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.839133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3268a-8d7e-4378-a268-a37990792189 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.872957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.873492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.873768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.874038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.874750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] RBAC: Adding query filter params (role.id=9245c00a908549cf9b22dd1e82d22bb2, scope.project.id=345b78d36cf14c2b87b6d80166b160b4, user.id=4ab519681e0b45818647b9c7690ca7c9) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:57.875055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.876507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec7915-3f86-4ab1-8992-cd3a708fee21 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.882034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 586/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/role_assignments?user.id=4ab519681e0b45818647b9c7690ca7c9&scope.project.id=345b78d36cf14c2b87b6d80166b160b4&role.id=9245c00a908549cf9b22dd1e82d22bb2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:59.776253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91170-3d81-4f97-80db-0f3fddc88946 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:59.778751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91170-3d81-4f97-80db-0f3fddc88946 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:59.778751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91170-3d81-4f97-80db-0f3fddc88946 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:59.778751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 545/1131] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:36:59.783511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:59.783951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:59.784052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:59.801629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:59.802823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:59.868416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 587/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:36:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:00.137997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed91170-3d81-4f97-80db-0f3fddc88946 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:00.236878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed91170-3d81-4f97-80db-0f3fddc88946 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:00.269703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:00.270329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:00.270608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:00.270877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:00.272046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:00.273481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0017d0-ca0a-4f03-9d1b-675a20ded14e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:00.275059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 546/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:00.281326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:00.357885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:00.377395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8f3e9-73cb-4c2e-9b11-0ad106a7600a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:00.409691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:00.410232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:00.410510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:00.410781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:00.411425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:00.412877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578a975-0343-432b-9afb-bbbc6a8f81fa None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:00.439056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 588/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:37:00 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:02.350186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ec8c-1f2a-46d4-9fb1-ee40197ed2d8 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:02.350748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ec8c-1f2a-46d4-9fb1-ee40197ed2d8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:02.350748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ec8c-1f2a-46d4-9fb1-ee40197ed2d8 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:02.352096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 547/1135] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:02.357999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:02.358406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:02.358549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:02.376623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:02.377814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:02.442950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 589/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:02.710224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4ec8c-1f2a-46d4-9fb1-ee40197ed2d8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:02.772118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:02.813681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4ec8c-1f2a-46d4-9fb1-ee40197ed2d8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:02.846366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:02.846982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:02.847256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:02.847542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:02.848638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:02.850047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb36a-a459-486e-80b5-0c37ba78732b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:02.851619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 548/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:02.857351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:02.917835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:02.955413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d153b5-ea77-42e1-bad2-86f21f63c90b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:02.987929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:02.988522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:02.988795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:02.989061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:02.989692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:02.991110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299cbb-0cc9-4734-9eea-eabcaf5fdbf0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:03.012618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 590/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:37:02 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:04.946344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9abe9-2c2d-476a-825a-b3e09fa6d890 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:04.947394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9abe9-2c2d-476a-825a-b3e09fa6d890 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:04.947394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9abe9-2c2d-476a-825a-b3e09fa6d890 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:04.948669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 549/1139] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:04.953997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:04.954378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:04.954534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:04.972842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:04.974017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:05.038403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 591/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:05.302387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9abe9-2c2d-476a-825a-b3e09fa6d890 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:05.403023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9abe9-2c2d-476a-825a-b3e09fa6d890 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:05.427350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:05.436132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:05.436754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:05.437032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:05.437305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:05.447721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:05.452384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:05.453564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:05.454492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 550/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:05.459409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:05.506846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:05.557121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8277ed-7741-473e-afcb-5fcdf617577c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:05.589941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:05.590542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:05.590812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:05.591075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:05.591742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:05.592037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:05.593387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:05.600973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 592/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:05.606937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:05.704264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b02f-79e4-4485-b3f8-5db9be675311 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:05.737635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:05.738250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:05.738532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:05.738807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] PATH_INFO: `/v3/users/d1f84c86dff644848de6a8bd9c422e48` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:05.752049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] RBAC: Authorizing `identity:get_user(user_id=d1f84c86dff644848de6a8bd9c422e48)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:05.753490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:05.755127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 551/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:05 2022] GET /identity/v3/users/d1f84c86dff644848de6a8bd9c422e48 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:05.761536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:05.857811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98674b0-c937-45e2-937e-3489c79d1f96 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:05.868042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:05.891860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:05.892462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:05.892739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:05.893013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:05.957553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:05.958988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:06.053641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 593/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:37:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:37:06.060125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:06.105894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:06.160039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fffa7-c6b5-4bf8-a79e-8bccf924852d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:06.193237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:06.193859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.194137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.194407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] PATH_INFO: `/v3/projects/50200a91e30441d596fa5a8867dc28d5` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.195533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] RBAC: Authorizing `identity:get_project(project_id=50200a91e30441d596fa5a8867dc28d5)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:06.197563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8c53-ab19-4fe7-969f-336c79f6ce7c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:06.199187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 552/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:06 2022] GET /identity/v3/projects/50200a91e30441d596fa5a8867dc28d5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:06.205555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:06.218077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:06.302610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c626c2b-a47d-4d5b-96d6-d028bd74f605 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:06.335310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:06.335930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.336206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.336475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.337179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=50200a91e30441d596fa5a8867dc28d5, user.id=d1f84c86dff644848de6a8bd9c422e48) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:06.338034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:06.339504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091de9-5aab-4b26-99b2-98c1fb196c19 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:06.344696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 594/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:37:06 2022] GET /identity/v3/role_assignments?user.id=d1f84c86dff644848de6a8bd9c422e48&scope.project.id=50200a91e30441d596fa5a8867dc28d5&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:08.221684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75330-04f3-4a28-b586-9806f7b1cad8 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.222243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75330-04f3-4a28-b586-9806f7b1cad8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.222243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b75330-04f3-4a28-b586-9806f7b1cad8 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.223619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 553/1147] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:08.228862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.229291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.229408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.247885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:08.249070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:08.313261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 595/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:08.591306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b75330-04f3-4a28-b586-9806f7b1cad8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.690478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b75330-04f3-4a28-b586-9806f7b1cad8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.703151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:08.723822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.724448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.724727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.724998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.735646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:08.740341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:08.741437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:08.742551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 554/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:08.748113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.751292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:08.844079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ecac-24b4-416a-bffa-2ea484621a23 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.877106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.877692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.877965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.878230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.878872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:08.879167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.880545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.888185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 596/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:08.893953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.996619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95db-ff53-4e0e-b39d-3fb2df88eaec None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:09.029887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:09.030508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.030787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.031058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] PATH_INFO: `/v3/users/d1f84c86dff644848de6a8bd9c422e48` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.032193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] RBAC: Authorizing `identity:get_user(user_id=d1f84c86dff644848de6a8bd9c422e48)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:09.033632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:09.035216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 555/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/users/d1f84c86dff644848de6a8bd9c422e48 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:09.041081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:09.105120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:09.139344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e17096-6151-4988-9d2e-f406a1130514 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:09.171820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:09.172434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.172706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.172972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] PATH_INFO: `/v3/projects/50200a91e30441d596fa5a8867dc28d5` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.174062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] RBAC: Authorizing `identity:get_project(project_id=50200a91e30441d596fa5a8867dc28d5)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:09.175501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86799121-3e68-4536-8798-2af48ae5fc83 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:09.177096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 597/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:09 2022] GET /identity/v3/projects/50200a91e30441d596fa5a8867dc28d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:09.183579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:09.224004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:09.282522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e45bb6-0515-41f0-aaf5-1cef07fa153b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:09.316459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:09.317104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.317376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.317641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] PATH_INFO: `/v3/projects/50200a91e30441d596fa5a8867dc28d5/users/d1f84c86dff644848de6a8bd9c422e48/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.319516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=50200a91e30441d596fa5a8867dc28d5, user_id=d1f84c86dff644848de6a8bd9c422e48, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:09.320926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeded636-6ff2-41eb-ae95-9f4c2053a2d3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:09.333920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 556/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:37:09 2022] PUT /identity/v3/projects/50200a91e30441d596fa5a8867dc28d5/users/d1f84c86dff644848de6a8bd9c422e48/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:11.196335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307b0d0-dd8a-4434-b18c-40350be6bb16 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.197487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307b0d0-dd8a-4434-b18c-40350be6bb16 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.197605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307b0d0-dd8a-4434-b18c-40350be6bb16 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.198857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 598/1154] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:11.204009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.204386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.204557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.222589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:11.223809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:11.288160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 557/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:11.555223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307b0d0-dd8a-4434-b18c-40350be6bb16 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:11.654737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307b0d0-dd8a-4434-b18c-40350be6bb16 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:11.687560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:11.688173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.688454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.688726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.690543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:11.701536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:11.706153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:11.707289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:37:11.708354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 599/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:11.713852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:11.742728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:11.810624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f72afe-f985-4462-bce7-43fecb4f378e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:11.842769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:11.843358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.843659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.843927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.844572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:11.844868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:11.846222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:11.854402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 558/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:11.860285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:11.954017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f650b-ffb7-4ede-9ad9-50626dee109f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:11.988226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:11.988851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.989133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.989406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] PATH_INFO: `/v3/users/d1f84c86dff644848de6a8bd9c422e48` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.990523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] RBAC: Authorizing `identity:get_user(user_id=d1f84c86dff644848de6a8bd9c422e48)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:11.991988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:11.994052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 600/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:11 2022] GET /identity/v3/users/d1f84c86dff644848de6a8bd9c422e48 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:11.999986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.088213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:12.096694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fadb4-3687-4c25-91be-2d38d1ea3305 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.130264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.130863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.131134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.131399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.196435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.197875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.283037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:12.293309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 559/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:37:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:37:12.299355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.358892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:12.397038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0b90-9213-4a36-b927-703d5edf9006 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.429052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.429661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.429937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.430206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] PATH_INFO: `/v3/projects/50200a91e30441d596fa5a8867dc28d5` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.431308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] RBAC: Authorizing `identity:get_project(project_id=50200a91e30441d596fa5a8867dc28d5)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.432750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef472113-1d13-4b53-91b6-51ef97917153 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.434333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 601/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:12 2022] GET /identity/v3/projects/50200a91e30441d596fa5a8867dc28d5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:12.440347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.535636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb72b7-d039-4e3f-a9f6-c57dc6c5ac1d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.568148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.568734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.569007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.569272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.569975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=50200a91e30441d596fa5a8867dc28d5, user.id=d1f84c86dff644848de6a8bd9c422e48) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:12.570274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.572395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5cf5c-c73a-4b24-90ab-8209ad2349e3 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.577912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 560/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:37:12 2022] GET /identity/v3/role_assignments?user.id=d1f84c86dff644848de6a8bd9c422e48&scope.project.id=50200a91e30441d596fa5a8867dc28d5&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:14.486905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc18311-e602-44d4-a7f4-c42747ed47d0 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:14.487466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc18311-e602-44d4-a7f4-c42747ed47d0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:14.487466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc18311-e602-44d4-a7f4-c42747ed47d0 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:14.488826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 602/1162] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:14.494561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:14.494951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:14.495100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:14.513318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:14.514501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:14.563289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:14.580922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 561/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:14.845653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc18311-e602-44d4-a7f4-c42747ed47d0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:14.945830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc18311-e602-44d4-a7f4-c42747ed47d0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:14.973758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:14.980307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:14.980901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:14.981179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:14.981448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.047318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.048788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:15.143152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 603/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:37:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:37:15.149001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:15.244029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c23c3-0a72-498f-8763-720a2eb8fb0f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:15.276611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:15.277223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.277496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.277763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.284169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.285596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-81e1eeb1-60f2-48f3-90d6-862abcbfd2e1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:15.297170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:15.298435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:15.299556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 562/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:37:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:15.304256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:15.333126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:15.401026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc403e-df79-4e69-91f9-166a68498b2d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:15.435790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:15.436812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.437319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.437757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.438459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:15.438710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.440143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90443710-e227-4b0d-89ef-90b743efb459 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:15.447329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 604/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Aug 24 17:37:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:17.310717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674077-e5d6-407e-92fa-696d5d7eecdb None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.311247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674077-e5d6-407e-92fa-696d5d7eecdb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.311247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674077-e5d6-407e-92fa-696d5d7eecdb None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.312650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 563/1167] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:17.317889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c56f2-61dc-47b4-90df-b9141977d7af None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.318311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c56f2-61dc-47b4-90df-b9141977d7af None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.318431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c56f2-61dc-47b4-90df-b9141977d7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.336488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ab0c56f2-61dc-47b4-90df-b9141977d7af None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:17.338267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ab0c56f2-61dc-47b4-90df-b9141977d7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.402219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 605/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:17.667250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674077-e5d6-407e-92fa-696d5d7eecdb None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.715777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:17.768357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674077-e5d6-407e-92fa-696d5d7eecdb None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.803634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.804235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.804513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.804783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.805940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.807316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3340-aeab-42e3-98dd-84dbc7edd78d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:17.823348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 564/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Wed Aug 24 17:37:17 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:19.707007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186708f9-3198-4079-b9d6-ca9f6de4711f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:19.707585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186708f9-3198-4079-b9d6-ca9f6de4711f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:19.707585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186708f9-3198-4079-b9d6-ca9f6de4711f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:19.708908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 606/1170] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:19.714088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:19.714489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:19.714637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:19.732866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:19.734309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:19.798387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 565/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:20.063617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186708f9-3198-4079-b9d6-ca9f6de4711f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.089024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:20.163039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186708f9-3198-4079-b9d6-ca9f6de4711f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.196491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.197106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.197384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.197654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] PATH_INFO: `/v3/domains/22071a3765504a3191c4695876fdbdcf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.207743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22071a3765504a3191c4695876fdbdcf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.209188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac618e8-37af-4b20-b4e3-a04767ff8779 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.210766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 607/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 24 17:37:20 2022] GET /identity/v3/domains/22071a3765504a3191c4695876fdbdcf => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:20.217249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.251292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:20.315819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e5ac3-963c-444a-848b-6f991ff2a5f0 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.348460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.349064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.349343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.349614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.350245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.351698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230751e3-0953-4305-bc0e-ad2ed19d7298 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.377937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 566/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:37:20 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:22.269407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7784-73c5-478a-9f32-19785761c6d6 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.269964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7784-73c5-478a-9f32-19785761c6d6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.269964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab7784-73c5-478a-9f32-19785761c6d6 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.271270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 608/1174] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:22.277208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.277617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.277761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.295995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:22.297186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:22.361220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 567/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:22.626956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab7784-73c5-478a-9f32-19785761c6d6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:22.709509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:22.726044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab7784-73c5-478a-9f32-19785761c6d6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:22.758789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:22.759425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.759731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.760001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] PATH_INFO: `/v3/domains/22071a3765504a3191c4695876fdbdcf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.761095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22071a3765504a3191c4695876fdbdcf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:22.762524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e31115-9ac4-42d6-bb75-5dd02c5d7d75 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:22.764143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 609/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Aug 24 17:37:22 2022] GET /identity/v3/domains/22071a3765504a3191c4695876fdbdcf => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:22.769830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:22.788315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:22.866875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48c2bc-92a8-452c-9c53-b59559350feb None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:22.900915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:22.901515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.901795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.902061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.902699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:22.904153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ef51-8c0d-4858-99be-5ca18e7adc95 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:22.926480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 568/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:37:22 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:24.880691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65b25-a8be-41c2-98c2-223560d32d46 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:24.881203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65b25-a8be-41c2-98c2-223560d32d46 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:24.881203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff65b25-a8be-41c2-98c2-223560d32d46 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:24.882513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 610/1178] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:24.888199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:24.888634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:24.888742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:24.907057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:24.908283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:24.972939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 569/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:37:25.247305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff65b25-a8be-41c2-98c2-223560d32d46 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:25.346059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff65b25-a8be-41c2-98c2-223560d32d46 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:25.378614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:25.379230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:25.379541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:25.379815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:25.390513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:25.395101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:25.396293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:37:25.397356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 611/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:25.402169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:25.405260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:25.498880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e981e65-c063-4ba9-95c7-f3bc8c3ed074 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:25.533926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:25.534530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:25.534807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:25.535077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:25.535768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:25.536069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:25.537435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:25.545393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 570/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:25.551169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:25.566381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:25.648463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee61ed-b12f-48f1-bada-091d6fd4b85a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:25.681351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:25.682063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:25.682347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:25.682620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] PATH_INFO: `/v3/users/b196e2d5e7ca4343a92365d8fe757c74` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:25.695851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] RBAC: Authorizing `identity:get_user(user_id=b196e2d5e7ca4343a92365d8fe757c74)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:25.697293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:25.698894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 612/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:25 2022] GET /identity/v3/users/b196e2d5e7ca4343a92365d8fe757c74 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:25.704830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:25.769241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:25.804025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae2b13-272f-4e13-a31c-df00b036edcd None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:25.836403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:25.837004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:25.837556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:25.837827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:25.902763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:25.904236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:25.965742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:26.001477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 571/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:37:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:37:26.007461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:26.073588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:26.102750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba5ba9-717b-470c-add8-a989df304602 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:26.134677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:26.135281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:26.135579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:26.135848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] PATH_INFO: `/v3/projects/3af5eef3b9e141ada6cd609d6f06e515` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:26.136933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] RBAC: Authorizing `identity:get_project(project_id=3af5eef3b9e141ada6cd609d6f06e515)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:26.138337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76263a-bcbc-4042-9653-ff1a4081e2d0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:26.139958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 613/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:26 2022] GET /identity/v3/projects/3af5eef3b9e141ada6cd609d6f06e515 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:26.146005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:26.243526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1c7c-3c04-499a-9d8f-4a48762ec333 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:26.277240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:26.277840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:26.278122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:26.278403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:26.279139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=3af5eef3b9e141ada6cd609d6f06e515, user.id=b196e2d5e7ca4343a92365d8fe757c74) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:26.279442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:26.280903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ed46b-23f5-42fa-8720-56b4cf78aa1e None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:26.286148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 572/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:37:26 2022] GET /identity/v3/role_assignments?user.id=b196e2d5e7ca4343a92365d8fe757c74&scope.project.id=3af5eef3b9e141ada6cd609d6f06e515&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:28.142181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965152-7f9f-41e7-ada8-b6c16387b4ca None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:28.142732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965152-7f9f-41e7-ada8-b6c16387b4ca None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:28.142732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965152-7f9f-41e7-ada8-b6c16387b4ca None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:28.144092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 614/1186] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:28.149070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:28.149505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:28.150190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:28.167900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:28.169077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:28.191788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:28.234808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 573/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:28.500822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965152-7f9f-41e7-ada8-b6c16387b4ca None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:28.600128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965152-7f9f-41e7-ada8-b6c16387b4ca None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:28.633539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:28.634151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:28.634428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:28.634696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:28.641296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:28.645296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:28.650491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:28.651608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:37:28.652590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 615/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:28.657607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:28.754185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f305593-3d9a-4362-b245-b0d84811b91b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:28.786960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:28.787576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:28.787862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:28.788135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:28.788789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:28.789092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:28.790987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:28.798719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 574/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:28.804615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:28.904410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6fb6-86f7-46e6-a769-816861f90b18 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:28.936623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:28.937241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:28.937519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:28.937792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] PATH_INFO: `/v3/users/b196e2d5e7ca4343a92365d8fe757c74` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:28.938901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] RBAC: Authorizing `identity:get_user(user_id=b196e2d5e7ca4343a92365d8fe757c74)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:28.940355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:28.941913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 616/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:28 2022] GET /identity/v3/users/b196e2d5e7ca4343a92365d8fe757c74 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:28.948217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:28.973964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:29.046640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68aa85-24c6-46f9-997c-5a84a41054f1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:29.079974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:29.080595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:29.080873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:29.081140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] PATH_INFO: `/v3/projects/3af5eef3b9e141ada6cd609d6f06e515` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:29.082238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] RBAC: Authorizing `identity:get_project(project_id=3af5eef3b9e141ada6cd609d6f06e515)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:29.083692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1473f-a4f4-4af1-bbca-35167adcf13d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:29.085336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 575/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:28 2022] GET /identity/v3/projects/3af5eef3b9e141ada6cd609d6f06e515 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:29.091535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:29.155131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:29.188705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317981ab-1d8d-419e-be3d-52cb1b9441ad None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:29.220908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:29.221539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:29.221812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:29.222079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] PATH_INFO: `/v3/projects/3af5eef3b9e141ada6cd609d6f06e515/users/b196e2d5e7ca4343a92365d8fe757c74/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:29.223950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] RBAC: Authorizing `identity:create_grant(project_id=3af5eef3b9e141ada6cd609d6f06e515, user_id=b196e2d5e7ca4343a92365d8fe757c74, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:29.225364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7c8b-d0f2-4c92-81e6-1dc035f59b81 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:29.237390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 617/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:37:29 2022] PUT /identity/v3/projects/3af5eef3b9e141ada6cd609d6f06e515/users/b196e2d5e7ca4343a92365d8fe757c74/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:31.103099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c98-9626-4b00-b4aa-690ef48ce3be None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:31.103687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c98-9626-4b00-b4aa-690ef48ce3be None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:31.103687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555c98-9626-4b00-b4aa-690ef48ce3be None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:31.105055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 576/1193] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:37:31.110482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:31.110925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:31.111028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:31.129046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:31.130216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:31.192966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 618/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:37:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:37:31.465483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555c98-9626-4b00-b4aa-690ef48ce3be None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:31.556198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:31.566907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555c98-9626-4b00-b4aa-690ef48ce3be None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:31.600085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:31.600709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:31.600980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:31.601245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:31.611699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:37:31.616291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:37:31.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:37:31.619177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 577/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:37:31.623908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:31.676162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:31.718404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecb1f9-bfd7-40c3-b8f9-f7d4e74e6559 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:31.751299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:31.751914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:31.752189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:31.752454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:31.753097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:31.753392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:31.754741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:31.762361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 619/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:31.768215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:31.866206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8b4e7-e2f9-4a0c-80f2-4f917808ac19 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:31.898923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:31.899559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:31.899841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:31.900110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] PATH_INFO: `/v3/users/b196e2d5e7ca4343a92365d8fe757c74` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:31.901219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] RBAC: Authorizing `identity:get_user(user_id=b196e2d5e7ca4343a92365d8fe757c74)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:31.902645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:31.904242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 578/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Aug 24 17:37:31 2022] GET /identity/v3/users/b196e2d5e7ca4343a92365d8fe757c74 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:31.909969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:32.007313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b547db-67f7-4421-942c-41ddc75f8e89 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:32.025679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:32.042703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:32.043292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:32.043597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:32.043865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:32.107947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:32.109401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:32.202410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 620/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:37:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:37:32.208594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:32.228035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:32.306000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060cfa9-104b-4f83-bfdb-ce9d9a032b2b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:32.338790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:32.339403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:32.339822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:32.340091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] PATH_INFO: `/v3/projects/3af5eef3b9e141ada6cd609d6f06e515` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:32.341190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] RBAC: Authorizing `identity:get_project(project_id=3af5eef3b9e141ada6cd609d6f06e515)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:32.342608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4bd49-95d4-400e-8256-bdb2aa1d475a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:32.344252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 579/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Aug 24 17:37:32 2022] GET /identity/v3/projects/3af5eef3b9e141ada6cd609d6f06e515 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:32.350339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:32.369172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:32.448822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6b6a1-5d19-4ed9-81d8-957336ddf53e None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:32.482310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:32.483160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:32.483436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:32.483731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:32.484438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=3af5eef3b9e141ada6cd609d6f06e515, user.id=b196e2d5e7ca4343a92365d8fe757c74) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:32.484742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:32.486155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486908f4-8c40-42d9-9a4b-2d9e3df8628d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:32.491776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 621/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:37:32 2022] GET /identity/v3/role_assignments?user.id=b196e2d5e7ca4343a92365d8fe757c74&scope.project.id=3af5eef3b9e141ada6cd609d6f06e515&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:42.334072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=57 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:42.489147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=59 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:27.436318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785a8ce-7bc2-45f7-997c-c7a9d743e5f8 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:27.436908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785a8ce-7bc2-45f7-997c-c7a9d743e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:27.436908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785a8ce-7bc2-45f7-997c-c7a9d743e5f8 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:27.438495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 580/1201] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:38:27.444064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:27.444584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:27.444584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:27.466261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:27.467707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:27.559827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 622/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:27.863620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785a8ce-7bc2-45f7-997c-c7a9d743e5f8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:28.012029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785a8ce-7bc2-45f7-997c-c7a9d743e5f8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:28.050775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:28.051441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:28.051796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:28.052007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:28.063181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:28.064674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:28.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:28.085043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:28.086024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:28.086024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:28.086024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:28.086024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 581/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:28.092921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:28.233630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedeba56-ee7a-475a-b660-b726237510ce None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:28.267043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:28.276107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:28.277036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:28.277524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:28.277869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:28.278587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:28.278895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:28.280330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260902f-890c-4d19-8489-9cb3461d23a5 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:28.295006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 623/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:28.304794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:28.373991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:28.435815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a734f1-015c-436e-942e-5030704dae2a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:28.488721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:28.495544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:28.495544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:28.495544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:28.495544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:28.495544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb7631-3667-44b1-bbd3-e99d047a3a7c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:28.522639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 582/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Aug 24 17:38:28 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:30.709592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04315ed-90e3-4a3a-86b1-e6bedc24e128 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:30.710676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04315ed-90e3-4a3a-86b1-e6bedc24e128 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:30.710676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04315ed-90e3-4a3a-86b1-e6bedc24e128 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:30.711811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 624/1206] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:30.717792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:30.718307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:30.718397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:30.739157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:30.740514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:30.819597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 583/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:31.107443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04315ed-90e3-4a3a-86b1-e6bedc24e128 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.222195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04315ed-90e3-4a3a-86b1-e6bedc24e128 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.259076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.259760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.260149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.260259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.272099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:31.277133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:31.278666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:31.279610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 625/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:31.284301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.327812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:31.405882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521af29d-38b0-4a71-adb1-457b45b18d9c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.453118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.453836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.454184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.454570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.455415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:31.455813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.457370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.466315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 584/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:31.471955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.533474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:31.592858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551e0eb-aaa5-4e86-8abf-6becf8d89cbe None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.632401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.633124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.633459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.633818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.652689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.654165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:31.669432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:31.670590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:31.671753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:31.671753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:31.671753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:31.671753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 626/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:31.676222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.782908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:31.786446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3647d0-a8c2-4326-aa92-2b3aa89e3c15 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.824253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.824876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.825153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.825424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.826076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:31.827928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.827928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.842284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 585/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:31.849433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.967913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf0968-ee37-4626-8a81-823f3f8830a9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.969734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.016136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.017013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.017482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.017739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.027423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.029184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:32.042323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:32.043342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:32.044346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 627/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:32.049713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.170861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29904b76-282f-4709-bc16-882ee511d371 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.218197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.218894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.219293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.219717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.221339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:32.221339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.222448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.244343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 586/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:32.250652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.388368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7d63b-3e63-44ff-b77e-22bca9e5b7c0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.430128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.430745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.432617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.432617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.467280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.515299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.518573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.632449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 628/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:38:32.638418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.661011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.773352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084ccbf-d658-4e4f-aa48-0dfdd90ca1f4 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.812508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.813179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.813406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.813675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.824218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.825733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:32.841444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:32.847285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:32.849009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:32.849009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:32.849009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:32.849009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 587/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:38:32.849305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.915267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.957342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d0269-221d-4fac-91f5-9a8f534ff34d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.994209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.994833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.995082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.995352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.996018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:33.000040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:33.000040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:33.010481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 629/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:33.016000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:33.102005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:33.136714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f13f5-5899-44f6-8956-a15ed32adcab None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:33.171757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:33.172407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:33.172680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:33.172941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:33.180620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:33.182047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:33.194336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:33.195279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:33.196261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 588/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:33.201293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:33.311851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a1a95-2145-44a0-8772-29347d78f866 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:33.338718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:33.351908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:33.352642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:33.352938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:33.353273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:33.354020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:33.354389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:33.356255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6350e1-96ba-4e82-8352-26ee9c3c3048 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:33.370361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 630/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:33.376291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:33.496622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d93e93-5ca5-4d41-9aa3-579b38ccbe15 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:33.538767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:33.545883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:33.546554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:33.546922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:33.547250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:33.548070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=cb029fc999c644408f292f85e4ab114b) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:33.548435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:33.549929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fc4f-aa28-4724-b3b9-ebe996c2ed16 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:33.556443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 589/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:38:33 2022] GET /identity/v3/role_assignments?user.id=cb029fc999c644408f292f85e4ab114b&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:35.494794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6407e-f7fe-4a52-8c69-31e93f6db5eb None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:35.495593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6407e-f7fe-4a52-8c69-31e93f6db5eb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:35.495593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6407e-f7fe-4a52-8c69-31e93f6db5eb None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:35.496662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 631/1220] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:35.502788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:35.503247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:35.503353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:35.522785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:35.524043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:35.605866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 590/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:38:35.878489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6407e-f7fe-4a52-8c69-31e93f6db5eb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:35.995693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6407e-f7fe-4a52-8c69-31e93f6db5eb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.039855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.040563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.040880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.041260 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.053071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:36.057777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:36.058757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:36.059679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 632/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:36.065201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.174002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b854d-4a36-48e4-beb0-b43c9a05639d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.203237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:36.215669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.216307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.216587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.216859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.217518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:36.217821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.219205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.228145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 591/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:36.234289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.280200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:36.343777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041c96f-bc74-45d4-9897-e9d3d88824d9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.380936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.381808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.382099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.382436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.398800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.400595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:36.415863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:36.416997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:36.417900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:36.417900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:36.417900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:36.417900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 633/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:36.422899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.539243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1af261-6fdc-4313-80fe-e0a26175acb5 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.576551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.577163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.577436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.577704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.578336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:36.578633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.580413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.598794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 592/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:36.600113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.704282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:36.711233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa837a8-01b3-4a0d-91af-17a444c4d179 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.752114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.752777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.753057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.753328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.761707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.763141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:36.775658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:36.776598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:36.778056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 634/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:38:36.783563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:36.835290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:36.899064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b86a4-01a8-4b8a-ab36-ceb5b3c5c4a1 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:36.939452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:36.940189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:36.940382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:36.940638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:36.942007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:36.942314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:36.943776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:36.968109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 593/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:36.969374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:37.088625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655ba1-bafd-4478-a780-81002ab8d863 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:37.126178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:37.126809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:37.127086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:37.127380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:37.138259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:37.139886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:37.156055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:37.157245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:37.158312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:37.158312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:37.158312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:37.158312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 635/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:37.164632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:37.278888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7df04-6c91-4145-a80e-75b032d5807b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:37.281516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:37.320771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:37.321473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:37.321786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:37.322093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:37.322843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:37.323206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:37.324712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:37.342357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 594/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:37.343644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:37.354069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:37.449246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff45efb-6f20-4560-814a-f4782c019983 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:37.484530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:37.485159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:37.485433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:37.485703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:37.494916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:37.496410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:37.508070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:37.509007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:37.509907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 636/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:37.517483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:37.630343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc580a-ee36-48cd-aca9-99b1891ab847 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:37.669897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:37.670668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:37.670951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:37.671227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:37.672187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:37.672500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:37.674382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68850e6-bbb6-4e17-83ab-7a375faa737a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:37.689999 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 595/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:37.696529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:37.773031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:37.809023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ea6-dffb-4dde-ad99-a2f9cbfd494b None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:37.845661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:37.846490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:37.846772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:37.847045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/cb029fc999c644408f292f85e4ab114b/roles/aa77b1f55770457780718c82fd9f2ea0` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:37.865021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=cb029fc999c644408f292f85e4ab114b, role_id=aa77b1f55770457780718c82fd9f2ea0)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:37.866502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d482c-1638-4001-8e9b-6d0712a6eec6 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:37.895594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 637/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:38:37 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/cb029fc999c644408f292f85e4ab114b/roles/aa77b1f55770457780718c82fd9f2ea0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:39.797414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27b608-3763-4d74-9115-d0490dc8472c None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:39.797934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27b608-3763-4d74-9115-d0490dc8472c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:39.797934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27b608-3763-4d74-9115-d0490dc8472c None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:39.799362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 596/1233] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:39.804823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:39.805299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:39.805444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:39.824880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:39.826241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:39.896770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 638/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:40.171434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27b608-3763-4d74-9115-d0490dc8472c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:40.203765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:40.285460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27b608-3763-4d74-9115-d0490dc8472c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:40.327320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:40.328016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:40.328323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:40.328673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:40.340465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:40.345919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:40.347185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:40.348784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 597/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Aug 24 17:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:40.354679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:40.408374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:40.467956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b04315-f765-43bf-8057-0e0f3c755942 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:40.507033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:40.507676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:40.507955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:40.508228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:40.508893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] RBAC: Adding query filter params (name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:40.509195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:40.510564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:40.518530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 639/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Aug 24 17:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:40.524596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:40.619773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:40.637543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877bd75-76a8-47be-b12e-c7c11352e95d None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:40.676323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:40.677014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:40.677298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:40.677570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:40.696144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:40.697935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:40.715442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:40.716542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:40.717445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:40.717445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:40.717445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:40.717445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 598/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:40.724207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:40.834491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8c495-fcc2-4158-ab2f-49e7caa7b745 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:40.844582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:40.871339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:40.871992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:40.872274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:40.872547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:40.873184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:40.873487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:40.875579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:40.889232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 640/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:40.896636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:41.013763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65617699-54b1-4961-a668-578b1b1ece93 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:41.056848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:41.057488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:41.057767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:41.058043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] PATH_INFO: `/v3/users/placement` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:41.060925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:41.066629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:41.068282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:41.079685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:41.081155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:41.082931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 599/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:38:41.087687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:41.192873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da227769-dc0b-448e-9a88-a7744ce377de None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:41.231868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:41.232497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:41.232780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:41.233037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:41.233750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:41.234047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:41.235461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:41.254036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 641/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:41.259885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:41.371954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a92477-71a0-44d2-8aad-e6cd153edabf None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:41.407350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:41.408007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:41.408288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:41.408593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:41.473630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:41.481794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:41.483382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:41.591242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 600/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:38:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:38:41.597230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:41.626257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:41.709309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a494d-7a10-4969-9a67-085e4a5059c9 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:41.746532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:41.747168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:41.747438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:41.747735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:41.757995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:41.759409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:41.774081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:41.775124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:41.776085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:41.776085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:41.776085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:41.776085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 642/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:41.781014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:41.882614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:41.888681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db82a-db9d-4329-9bf0-ff427a3f28de None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:41.924088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:41.924811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:41.924988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:41.925298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:41.925991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:41.926296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:41.927679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:41.939515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 601/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:41.945498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:42.042269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:42.066564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8072ced-3073-4ed4-a9af-a625622c9fcf None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:42.104479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:42.105192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.105497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.105767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.112471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:42.113935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:42.124785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:42.125851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:42.127119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 643/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:42.132342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:42.244539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30acc07-fe3b-4ab9-b635-85ec65aef99c None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:42.282388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:42.282998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.283275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.283604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.284380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:42.285295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:42.287047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a15c94-9f9c-43bc-afbe-43126ef342e1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:42.305729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 602/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:42.307232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:42.421981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a55c7-234f-4274-9f58-692e9458b05c None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:42.461270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:42.462001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:42.462349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:42.462753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:42.463691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] RBAC: Adding query filter params (role.id=aa77b1f55770457780718c82fd9f2ea0, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=cb029fc999c644408f292f85e4ab114b) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:42.463952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:42.465643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25447b5-2be6-46bb-8902-ecd41d558906 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:42.471977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 644/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:38:42 2022] GET /identity/v3/role_assignments?user.id=cb029fc999c644408f292f85e4ab114b&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=aa77b1f55770457780718c82fd9f2ea0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:44.547853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c1a0-35a5-4ecc-888a-4e424ac9fd93 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:44.548410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c1a0-35a5-4ecc-888a-4e424ac9fd93 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:44.548410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35c1a0-35a5-4ecc-888a-4e424ac9fd93 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:44.549807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 603/1247] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:44.556722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:44.557306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:44.557571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:44.564969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:44.579759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:44.581071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:44.663424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 645/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:44.936798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35c1a0-35a5-4ecc-888a-4e424ac9fd93 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:44.956920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:45.058218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35c1a0-35a5-4ecc-888a-4e424ac9fd93 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.099127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.100450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.100898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.101514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.115638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:45.121467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:45.122546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:45.124138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 604/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:45.129644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.243023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58f864-1bc7-4b36-a220-a41983b96beb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.283248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:45.291571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.291571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.291571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.291571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.291966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:45.292141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:45.293704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:45.303233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 646/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:45.309569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.406589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:45.436401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d21bd-5a8b-4efd-aa8a-4274db3f85dc None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.482174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.483058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.483354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.483657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.496282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:45.498102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:45.515425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:45.516635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:45.517539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:45.517539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:45.517539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:45.517539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 605/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:38:45.523797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.635386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe93b-f196-49a1-b72f-4073244bff0a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.683658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.684271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.684614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.684887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.685663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:45.685960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:45.687401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:45.701181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 647/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:45.707037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.826461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ab956-d064-4d11-948a-0c0c2d32041b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:45.855229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:45.864921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:45.865554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:45.865829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:45.866095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:45.873644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:45.875081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:45.886518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:45.887610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:45.889079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 606/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:45.893418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:45.950715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:46.004038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d49fc-99e9-4937-b5b7-5a9afdcadba7 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:46.043987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:46.044744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:46.045007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:46.045298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:46.046092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:46.046291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:46.047752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:46.070839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 648/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:46.072153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:46.189318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6fce3-3b5b-407f-a31d-002dfa36e7ea None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:46.230229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:46.230833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:46.231141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:46.231437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:46.295421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:46.310795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:46.312252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:46.418114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 607/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:38:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:38:46.424070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:46.529898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaeae6-cc1f-44fd-9650-cbae45befe56 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:46.544599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:46.569195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:46.570007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:46.570378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:46.570743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:46.581573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:46.583027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:46.600468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:46.601636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:46.602634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:46.602634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:46.602634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:46.602634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 649/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:38:46.607771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:46.703609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:46.716211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c80f-b0b4-4c2e-a47d-65f6a80328c8 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:46.752572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:46.753459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:46.753838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:46.754306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:46.755138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:46.755570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:46.757109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:46.769362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 608/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:46.775992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:46.881944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c84e-b7ad-49d1-ac6b-e17e349a247f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:46.916012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:46.916640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:46.916914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:46.917180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:46.923977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:46.925453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:46.937079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:46.938169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:46.939070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 650/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:46.945369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:47.062830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850719da-31cc-4a42-96c6-6bd0c7cc0e57 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:47.100872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:47.101609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:47.101924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:47.102229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:47.103063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:47.103373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:47.105373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c9813-c466-43b6-9dc8-0e456f9b236f None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:47.120394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 609/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:47.126585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:47.154764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:47.236723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71295586-5a01-4233-811e-db7f8ceb3e5d None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:47.275210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:47.275837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:47.276102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:47.276363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:47.277079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=cb029fc999c644408f292f85e4ab114b) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:47.277417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:47.278806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d94a1e-0771-43e9-a121-a383f602defd None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:47.284938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 651/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:38:47 2022] GET /identity/v3/role_assignments?user.id=cb029fc999c644408f292f85e4ab114b&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:49.202661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0be24e-d4e2-455c-b487-9615d2a2e730 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.203231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0be24e-d4e2-455c-b487-9615d2a2e730 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.203231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0be24e-d4e2-455c-b487-9615d2a2e730 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.204600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 610/1261] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:49.210157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.210525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.210703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.229245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:49.230433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:49.296509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 652/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:49.561605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0be24e-d4e2-455c-b487-9615d2a2e730 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.573017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:49.663859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0be24e-d4e2-455c-b487-9615d2a2e730 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.697320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:49.697938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.698218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.698491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.708494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:49.713135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:49.714229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:49.715437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 611/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:38:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:49.720171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.739387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:49.817666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146f0ca-d4a1-4336-b5cc-4862d60f12ef None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.850497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:49.851089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.851364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.851663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.852314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:49.852611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:49.854214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:49.862052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 653/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:38:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:49.867643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.932220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:49.964976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c86e-0bdb-4045-ba35-e7b80e8515af None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.999430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.000112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.000390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.000660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.012793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.014240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:50.029132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:50.030319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:50.031449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:50.031449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:50.031449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:50.031449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 612/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:50.036553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.117854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:50.136999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6632c63-cfa2-4043-b5a4-33137869f1cc None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.168896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.169491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.169768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.170039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.170665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:50.170962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.172351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.184705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 654/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:50.191164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.290015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273818-8c4e-4bee-acfe-07ceb4e3b382 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.309052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:50.323061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.323709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.323986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.324257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] PATH_INFO: `/v3/users/placement` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.332066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.333499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:50.344158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:50.345257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:50.346478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 613/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:50.351666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.447277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072faf7f-2ada-4340-880d-2f113e44ae33 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.479605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.480192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.480463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.480725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.481425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:50.481719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.483120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.491928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:50.500154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 655/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:50.506257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.604980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d0205-cbfd-45d0-884f-7c0f176f0fed None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.638424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.639043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.639314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.639609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.649667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.651090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:50.664926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:50.666030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:50.667324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:50.667324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:50.667324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:50.667324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 614/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:50.671858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.766485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92c526-e85f-46de-9ea0-1da4b22215b2 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.799762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.800320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.800595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.800863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.801490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:50.801788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.803143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.814069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 656/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:50.819572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.851631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:50.918530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2541cc-92fd-4706-8708-a8e648c191a6 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.952491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.953108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.953383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.953650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.960155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.961584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:50.974534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:50.975609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:50.976722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 615/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:50.983833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.047412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:51.088747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2ae6-b6e6-42b3-ae39-29b4a28b353f None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.121286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.121886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.122165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.122436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.123107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:51.123407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.124886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b72d4-f0ec-4bf7-8f4f-b2c1d6372700 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.136867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 657/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:51.142979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.232198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:51.240894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c0291-b6cb-4034-b50d-52a5ff70654a None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.275315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.275997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.276278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.276549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] PATH_INFO: `/v3/projects/ff81e911885e43ed86627df1e1a85893/users/cb029fc999c644408f292f85e4ab114b/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.278443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff81e911885e43ed86627df1e1a85893, user_id=cb029fc999c644408f292f85e4ab114b, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.279911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ddd17-1884-4c27-a013-621cdf2f31f5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.293796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 616/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Aug 24 17:38:51 2022] PUT /identity/v3/projects/ff81e911885e43ed86627df1e1a85893/users/cb029fc999c644408f292f85e4ab114b/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:53.202852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8da4-4652-4662-8ff0-d3d8175377ae None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:53.203384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8da4-4652-4662-8ff0-d3d8175377ae None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:53.203384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8da4-4652-4662-8ff0-d3d8175377ae None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:53.204761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 658/1274] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:53.209916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784007-a86c-4387-b986-3a6965e5322e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:53.210331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784007-a86c-4387-b986-3a6965e5322e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:53.210469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14784007-a86c-4387-b986-3a6965e5322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:53.228220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-14784007-a86c-4387-b986-3a6965e5322e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:53.229676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-14784007-a86c-4387-b986-3a6965e5322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:53.297051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 617/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:53.567906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8da4-4652-4662-8ff0-d3d8175377ae None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:53.669844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8da4-4652-4662-8ff0-d3d8175377ae None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:53.687211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:53.703896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:53.704520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:53.704799 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:53.705071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:53.714919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:38:53.719538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:53.720633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:53.722295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 659/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Aug 24 17:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:53.727143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784007-a86c-4387-b986-3a6965e5322e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:53.805849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:53.824774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14784007-a86c-4387-b986-3a6965e5322e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:53.858572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:53.859171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:53.859450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:53.859752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:53.860408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:53.860711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:53.862116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:53.869900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 618/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Aug 24 17:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:53.875519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:53.972650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57033324-0ce6-42aa-8d4c-2af10c442bfa None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.009814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.010577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.010959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.011132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.021632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.023074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:54.037321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:54.038245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:54.039196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:54.039196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:54.039196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:54.039196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 660/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:54.044365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:54.143039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d3e5b-9cdf-4546-85c9-71bb4b5bb93b None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.175922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.176513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.176790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.177061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.177691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:54.177991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.179350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.191186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 619/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:54.196798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:54.226743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:54.295191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc137201-285b-4ba8-95e3-a24a1ee262e8 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.328607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.329227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.329508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.329781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.337646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.339077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:38:54.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:54.350991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:54.352345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 661/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:54.357042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:54.360799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:54.454611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba76bd-5547-40d6-977e-6d0586decac2 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.487791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.488379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.488651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.488915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.489618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:54.489916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.491332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.507602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 620/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:54.513322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:54.605569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:54.616029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416312d2-49d0-4b60-9b82-72bdd81b7d15 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.649151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.649753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.650033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.650303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.714919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.716404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.801327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:54.811224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 662/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:38:54.817796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:54.877123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:54.915682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d649e-b4e2-42d3-a109-2ff5885a98c5 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:54.948707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:54.949317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:54.949590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:54.949853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:54.959916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:54.961339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:38:54.975120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:54.976352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:38:54.977367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:38:54.977367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:38:54.977367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:54.977367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 621/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:54.982439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:55.083241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393517ee-c63d-4fd2-a060-8cc95336b0a0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:55.117347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:55.117936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:55.118209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:55.118474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:55.119091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:55.119387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:55.120760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:55.131399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 663/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Aug 24 17:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:55.137287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:55.234876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e4bbf-0925-48f5-9316-b8d8ce4d6477 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:55.248597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:55.268881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:55.269500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:55.269780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:55.270052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:55.276577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:55.278014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:38:55.289620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:55.290722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: ERROR keystone.server.flask.application Aug 24 17:38:55.291836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 622/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Aug 24 17:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:55.296696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:55.345742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:55.395773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587a9a7-4bb9-40dc-83cc-0395796f4ca4 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:55.427926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:55.428519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:55.428797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:55.429069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:55.429744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:55.430044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:55.431469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d95443-d51c-462f-9264-52bfb67a8b26 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:55.443521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 664/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Aug 24 17:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:55.449602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:55.548430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02c71f-07f7-4739-81f1-37d1e3db2c80 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:55.580276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:55.581396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:55.581673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:55.581941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:55.582655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] RBAC: Adding query filter params (role.id=bc2f8d451570479d9dd920a4c92b8ddf, scope.project.id=ff81e911885e43ed86627df1e1a85893, user.id=cb029fc999c644408f292f85e4ab114b) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:55.582962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:55.584418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48303-08da-4dc4-9f79-63f12fa3f951 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:55.590025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 623/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Aug 24 17:38:55 2022] GET /identity/v3/role_assignments?user.id=cb029fc999c644408f292f85e4ab114b&scope.project.id=ff81e911885e43ed86627df1e1a85893&role.id=bc2f8d451570479d9dd920a4c92b8ddf => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:57.492853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfcee-a73c-4285-8f87-b9c8fe0f5614 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:57.493421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfcee-a73c-4285-8f87-b9c8fe0f5614 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:57.493421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfcee-a73c-4285-8f87-b9c8fe0f5614 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:57.494727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 665/1288] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:38:57.499971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:57.500403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:57.500512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:57.518786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:57.520028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:57.549451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:57.586110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 624/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:38:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:38:57.854318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfcee-a73c-4285-8f87-b9c8fe0f5614 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:57.952766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfcee-a73c-4285-8f87-b9c8fe0f5614 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:57.963737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:57.989577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:57.990221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:57.990509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:57.990778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:57.991457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:57.992963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:38:58.002556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:38:58.004043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:38:58.005355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 666/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:38:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:38:58.011228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:58.121053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ad482-32e0-4173-ad76-aa731373169c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:58.152990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:58.153591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:58.153865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:58.154133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:58.154746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:58.155044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:58.156435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa57f0-d28d-44ac-a96b-2830446e5c39 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:58.161647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 625/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 17:38:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:58.167425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:58.264795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9882e-207f-45ac-a48a-6bdde8d58946 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:58.296386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:58.296980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:58.297258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:58.297531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] PATH_INFO: `/v3/services` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:58.298145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:58.298446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:58.300353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f1aa5-0731-4534-8ca9-340b9ed98446 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:58.304610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 667/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 17:38:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:00.166418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a462-81db-4180-956e-6bfb9d4ae0f6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.166945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a462-81db-4180-956e-6bfb9d4ae0f6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.166945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a462-81db-4180-956e-6bfb9d4ae0f6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.168348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 626/1293] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:39:00.173388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c796-84cd-453e-8b8e-349168579bcb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.173782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c796-84cd-453e-8b8e-349168579bcb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.173996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c796-84cd-453e-8b8e-349168579bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.192108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e5e6c796-84cd-453e-8b8e-349168579bcb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:00.193287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e5e6c796-84cd-453e-8b8e-349168579bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:00.221922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:00.258426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 668/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:39:00.523375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33a462-81db-4180-956e-6bfb9d4ae0f6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:00.556073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:00.626542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33a462-81db-4180-956e-6bfb9d4ae0f6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:00.660564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:00.661166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.661444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.661711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.662357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:00.663747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fc55b-ecef-490c-b688-0bcaab82436d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:00.678207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 627/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:02.554867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed77d-fda3-4850-b465-ffec77b16dfe None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.555351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed77d-fda3-4850-b465-ffec77b16dfe None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.555443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed77d-fda3-4850-b465-ffec77b16dfe None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.556764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 669/1296] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:39:02.562076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.562474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.562621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.581314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:02.582503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:02.652692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 628/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:39:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:39:02.915145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ed77d-fda3-4850-b465-ffec77b16dfe None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:03.019155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ed77d-fda3-4850-b465-ffec77b16dfe None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:03.057367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:03.057990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:03.058317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:03.058542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] PATH_INFO: `/v3/services/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:03.059196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:03.060615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:39:03.069907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:39:03.071132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:03.072200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 670/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:39:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:39:03.077088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:03.094890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:03.176607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d5f1b-a023-40bb-a627-7cf884ca650b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:03.208774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:03.209377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:03.209656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:03.209930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:03.210550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:03.210852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:03.212246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:03.217241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 629/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 17:39:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:03.223699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:03.255820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:03.322549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dba567-f4e4-4045-b971-014358c6eb0a None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:03.356316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:03.356906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:03.357181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:03.357450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:03.358134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] RBAC: Adding query filter params (interface=public, service_id=4c4d2706640f4c61b93b496bf01b1d83, region_id=RegionOne) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:03.358434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:03.359821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b9feb-da05-4596-9517-def3d31eb7ec None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:03.365220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 671/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Aug 24 17:39:03 2022] GET /identity/v3/endpoints?service_id=4c4d2706640f4c61b93b496bf01b1d83&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:03.370791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:03.451544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:03.469477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b019-5f7a-4e9f-b057-83117c17a766 None admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:03.503127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:03.503746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:03.504019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:03.504283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:03.504881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:03.506239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be6e02-672a-4fe4-a175-11aa111fe35d None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:03.512891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 630/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Aug 24 17:39:03 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:05.375214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3622-dd4c-4a99-bb56-27611e2841da None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:05.375780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3622-dd4c-4a99-bb56-27611e2841da None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:05.375780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd3622-dd4c-4a99-bb56-27611e2841da None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:05.377671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 672/1302] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:39:05.382741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:05.383157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:05.383276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:05.401298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:05.402471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:05.468069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 631/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:39:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:39:05.734213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd3622-dd4c-4a99-bb56-27611e2841da None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:05.827422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:05.834686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd3622-dd4c-4a99-bb56-27611e2841da None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:05.867840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:05.868459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:05.868736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:05.869006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:05.869580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:05.870937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:39:05.879975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:39:05.881147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application with Lock( Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: ERROR keystone.server.flask.application Aug 24 17:39:05.882173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 673/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Aug 24 17:39:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:39:05.887045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:05.959898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:05.982566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4c216-d4a4-4e1b-8d43-361a04519841 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:06.016118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:06.016659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:06.016938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:06.017211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] PATH_INFO: `/v3/services` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:06.017835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:06.018137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:06.019967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f65f-369b-4fd4-a0bc-596f25e303f1 None admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:06.026721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 632/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Aug 24 17:39:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:06.032436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:06.128030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae04ac-f351-43e5-9a97-7ce3c5e7b1e0 None admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:06.160166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:06.160753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:06.161028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:06.161294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:06.161933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:06.163272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbf57-55d7-47dd-9597-3e1f0907fe84 None admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:06.182547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 674/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:16.022567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=30 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:16.178285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:10.029255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73205a-807d-4f93-aedb-a1160a616e09 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.029838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73205a-807d-4f93-aedb-a1160a616e09 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.029838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73205a-807d-4f93-aedb-a1160a616e09 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.036435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9649-e28d-4826-a506-cdf33718cfaa None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.036899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9649-e28d-4826-a506-cdf33718cfaa None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.037022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b9649-e28d-4826-a506-cdf33718cfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.039793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 633/1307] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:40:10.111834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-285b9649-e28d-4826-a506-cdf33718cfaa None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.113046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-285b9649-e28d-4826-a506-cdf33718cfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.245795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 675/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:10.540220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.540795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.541257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.543195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 634/1309] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 24 17:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:10.549934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220d45c-7b78-43fa-be38-8d92620a299f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.550214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220d45c-7b78-43fa-be38-8d92620a299f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.550490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220d45c-7b78-43fa-be38-8d92620a299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.631337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b220d45c-7b78-43fa-be38-8d92620a299f None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:10.632583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b220d45c-7b78-43fa-be38-8d92620a299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.654065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:10.713050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 676/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 24 17:40:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:10.721722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:10.723465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:10.850912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761d9f-a4fb-46e7-a72b-05f00226623a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:10.936250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:10.936981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:10.937259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:10.937521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:10.939198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:11.000675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:11.059666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:11.061103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72500874-0d35-4440-8365-97a6348bde17 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:11.081020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 635/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:13.269509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:13.270063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:13.270063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:13.271390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 677/1312] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:13.277005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ae9-96fd-4a17-b909-3e86b8d2fc47 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:13.277519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ae9-96fd-4a17-b909-3e86b8d2fc47 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:13.277685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7ae9-96fd-4a17-b909-3e86b8d2fc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:13.367709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d44f7ae9-96fd-4a17-b909-3e86b8d2fc47 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:13.368939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d44f7ae9-96fd-4a17-b909-3e86b8d2fc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:13.468882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 636/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:13.751662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:13.753225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:13.857737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e69024-192b-4b83-b3a1-642961d414d4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:13.927802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:13.930839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:13.931668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:13.932138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:13.932539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:13.934885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.044388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:14.045951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707adf-1ce7-4024-9b68-003c1f78f645 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:14.064489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 678/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:16.296035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.296590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.296590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.297931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 637/1315] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:16.303083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2d6-c1d1-4f43-8e7f-4ccb07f52457 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.303517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2d6-c1d1-4f43-8e7f-4ccb07f52457 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.303657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a2d6-c1d1-4f43-8e7f-4ccb07f52457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.370768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b480a2d6-c1d1-4f43-8e7f-4ccb07f52457 None None] MFA Rules not processed for user `9c01f65b4421415cbf105fa576ebed73`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:16.372058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b480a2d6-c1d1-4f43-8e7f-4ccb07f52457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.423944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:16.471968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 679/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:16.756794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:16.758331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.768780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:16.861359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d87ea8-e45f-41b8-901b-eaaafabd504f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:16.932543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:16.933146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:16.933418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:16.933708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:16.935836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:16.999501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:17.046134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:17.047536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f47e5-30ea-4fda-9445-96b336278bb8 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:17.065421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 638/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:19.470258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.470854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.470854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.472176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 680/1318] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:19.477805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8733-7840-44ff-b603-6e8ec7cf5100 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.478218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8733-7840-44ff-b603-6e8ec7cf5100 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.478337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8733-7840-44ff-b603-6e8ec7cf5100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.543889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-88ef8733-7840-44ff-b603-6e8ec7cf5100 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:19.545105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-88ef8733-7840-44ff-b603-6e8ec7cf5100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:19.640252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 639/1319] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:19.920521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.922064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:20.026362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7317a5-c5cc-4bb2-9d7e-547b421cda4e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.097074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.097674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.097952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.098221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.100106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:20.107797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:20.207513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.208877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250347f3-512d-4031-bae2-5ef7e76de986 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.227312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 681/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:22.762353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:22.762917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:22.762917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:22.764299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 640/1321] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:22.769607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504677-f3c5-43b8-b03c-b14977612339 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:22.770038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504677-f3c5-43b8-b03c-b14977612339 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:22.770152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70504677-f3c5-43b8-b03c-b14977612339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:22.835244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-70504677-f3c5-43b8-b03c-b14977612339 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:22.836397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-70504677-f3c5-43b8-b03c-b14977612339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:22.895397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:22.937414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 682/1322] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:23.217027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:23.218595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:23.239442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:23.322708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7458bf-5025-42c3-a75a-5b9c9722df9d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:23.395940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:23.396564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:23.396833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:23.397103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:23.398699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:23.470745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:23.508733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:23.510180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed246f4-feb2-4f37-ae2d-521641f636c7 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:23.530852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 641/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:25.518575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741358d-0054-4542-900b-82366673ff44 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.519067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741358d-0054-4542-900b-82366673ff44 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.519067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741358d-0054-4542-900b-82366673ff44 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.520429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 683/1324] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:25.525664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c752-5a59-400b-b8cf-4d8307373b1e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:25.526100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c752-5a59-400b-b8cf-4d8307373b1e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:25.526236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c752-5a59-400b-b8cf-4d8307373b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:25.592455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b2c2c752-5a59-400b-b8cf-4d8307373b1e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:25.593665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b2c2c752-5a59-400b-b8cf-4d8307373b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:25.691128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 642/1325] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:25.972356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741358d-0054-4542-900b-82366673ff44 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:25.974146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4741358d-0054-4542-900b-82366673ff44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:26.084778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741358d-0054-4542-900b-82366673ff44 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:26.164766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:26.165431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:26.165712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:26.165981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:26.167587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:26.185180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:26.276699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:26.278905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803f7f4-79f9-45a8-bfae-f132be1b1e8f service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:26.297063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 684/1326] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:28.253882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.254432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.254432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.255787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 643/1327] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:28.262168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9b9be-1a5e-427c-8629-2a81c377b7a1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.262775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9b9be-1a5e-427c-8629-2a81c377b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.262775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9b9be-1a5e-427c-8629-2a81c377b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.331539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-84b9b9be-1a5e-427c-8629-2a81c377b7a1 None None] MFA Rules not processed for user `9c01f65b4421415cbf105fa576ebed73`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:28.332773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-84b9b9be-1a5e-427c-8629-2a81c377b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.404192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:28.437267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 685/1328] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:28.738455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:28.740736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:28.784969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:28.869963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63622fc5-a0c1-41a6-8daf-84d551d0b407 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:28.944216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:28.944824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:28.945102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:28.945372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:28.947024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:29.029542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:29.060926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:29.062313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266aef9-fed4-4062-88bb-252f3ba42e08 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:29.080808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 644/1329] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:31.231827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.232395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.232395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.233741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 686/1330] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:31.239017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d137b-4c57-45fc-8302-b4b6acd0ff2a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.239432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d137b-4c57-45fc-8302-b4b6acd0ff2a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.239572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d137b-4c57-45fc-8302-b4b6acd0ff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.305875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6e5d137b-4c57-45fc-8302-b4b6acd0ff2a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:31.307094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6e5d137b-4c57-45fc-8302-b4b6acd0ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:31.401979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 645/1331] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:31.679251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:31.680891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:31.786182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350317-aac6-4f98-b788-4d26ccf419f4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:31.862261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:31.862869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:31.863142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:31.863415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:31.865051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:31.890967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:31.971166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:31.972633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682cb18-a167-4c8d-a6ea-d1103f882f7d service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:31.991284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 687/1332] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:33.941226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.941736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.941736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.943053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 646/1333] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:33.948832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478689-169d-46a1-ac1a-cee96c6a07c7 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.949254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478689-169d-46a1-ac1a-cee96c6a07c7 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.949381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478689-169d-46a1-ac1a-cee96c6a07c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.017825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-fd478689-169d-46a1-ac1a-cee96c6a07c7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:34.019023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fd478689-169d-46a1-ac1a-cee96c6a07c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.092127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.115121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 688/1334] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:34.398363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.399934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.440976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.504264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10f7c3-07bc-47cb-b7ca-ae2e8cfd88dd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:34.576212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:34.576830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.577109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.577379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.579019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.661280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.686974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.688493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a63cd-279f-44de-a24c-e40b961d24de service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:34.707112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 647/1335] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:36.924096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.924652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.924652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.925915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 689/1336] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:36.931304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99725-1a77-41de-ab5b-df78476552eb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.931783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99725-1a77-41de-ab5b-df78476552eb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.931866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd99725-1a77-41de-ab5b-df78476552eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.997731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1dd99725-1a77-41de-ab5b-df78476552eb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.998932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1dd99725-1a77-41de-ab5b-df78476552eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.096534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 648/1337] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:37.387469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.389068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.491796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811e388-e68e-4d6a-be64-9702a2370d60 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.566458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.567059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.567334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.567630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.569242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.605909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:37.679251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.680665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781506a-b649-4a06-9b85-f80df7454987 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.699257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 690/1338] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:41.488863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:41.489370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:41.489467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:41.490755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 649/1339] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:41.496676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938e9d-09dc-457d-9333-28dec8ed5b31 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:41.497124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938e9d-09dc-457d-9333-28dec8ed5b31 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:41.497232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938e9d-09dc-457d-9333-28dec8ed5b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:41.567368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9c938e9d-09dc-457d-9333-28dec8ed5b31 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:41.568656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9c938e9d-09dc-457d-9333-28dec8ed5b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:41.658058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:41.670952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 691/1340] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:41.954851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:41.956460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.010376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:42.066769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9483d-335e-4f66-ab96-8a1533d04273 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.140142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.140745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.141015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.141278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.142845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.233199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:42.250574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.252065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbaffe-9d15-4bcc-b887-aa5a78e89b0d service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.270637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 650/1341] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:44.827704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.828220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.828220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.829634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 692/1342] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:44.835275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732e48-6022-487e-ad21-ccd11f875c21 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:44.835873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732e48-6022-487e-ad21-ccd11f875c21 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:44.835873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732e48-6022-487e-ad21-ccd11f875c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:44.904162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1a732e48-6022-487e-ad21-ccd11f875c21 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:44.905373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a732e48-6022-487e-ad21-ccd11f875c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.011119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 651/1343] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:45.293767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:45.295363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.399304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fa637-3905-4591-9180-559a49b6fdd7 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:45.474045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:45.474673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:45.474952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:45.475221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:45.476973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:45.525005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:45.593805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:45.595161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0b986-4e57-4490-901b-7e2645c460d6 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:45.614591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 693/1344] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:49.494264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.494820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.494820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.496179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 652/1345] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:49.501518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85772d-d4ef-4a52-81b0-e8579ec0e920 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.501923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85772d-d4ef-4a52-81b0-e8579ec0e920 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.502060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85772d-d4ef-4a52-81b0-e8579ec0e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.589189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-bc85772d-d4ef-4a52-81b0-e8579ec0e920 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:49.590416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-bc85772d-d4ef-4a52-81b0-e8579ec0e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:49.708054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:49.711464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 694/1346] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:49.996715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:49.998251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:50.067679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:50.127831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be63d3-7436-4027-bbd3-5eb40efbb4cf None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.205041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.205648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.205924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.206193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.207803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:50.319167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:50.323889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.325977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc86d7e-1dbe-4233-803d-f0d4b4f31437 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.344817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 653/1347] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:55.288920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40295b5-d3d1-43bb-9d28-265ffc078b8c None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:55.289422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40295b5-d3d1-43bb-9d28-265ffc078b8c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:55.289515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40295b5-d3d1-43bb-9d28-265ffc078b8c None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:55.292806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 695/1348] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:55.301287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca0-d17a-4bc4-90ab-73db59efa8c0 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:55.301706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca0-d17a-4bc4-90ab-73db59efa8c0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:55.302143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98ca0-d17a-4bc4-90ab-73db59efa8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:55.416551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c4d98ca0-d17a-4bc4-90ab-73db59efa8c0 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:55.418149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c4d98ca0-d17a-4bc4-90ab-73db59efa8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:55.578028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 654/1349] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:40:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:59.978163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427909a-f607-40e2-a9aa-e4cbafd32409 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.978721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427909a-f607-40e2-a9aa-e4cbafd32409 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.978721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427909a-f607-40e2-a9aa-e4cbafd32409 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:59.980246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 696/1350] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:59.985061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f574e-30b1-4be4-9aaa-cce06a95486f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:59.985533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f574e-30b1-4be4-9aaa-cce06a95486f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:59.985811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f574e-30b1-4be4-9aaa-cce06a95486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:00.045256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:00.074014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-505f574e-30b1-4be4-9aaa-cce06a95486f None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:00.075221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-505f574e-30b1-4be4-9aaa-cce06a95486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:00.189907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 655/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:40:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:01.324458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e6d8-dd89-46c1-94a1-a526322edb93 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.325006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e6d8-dd89-46c1-94a1-a526322edb93 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.325096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2e6d8-dd89-46c1-94a1-a526322edb93 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.326548 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 697/1352] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:01.331831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2329b2-c463-4dad-98ce-4d214be0aa24 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.332227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2329b2-c463-4dad-98ce-4d214be0aa24 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.332387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2329b2-c463-4dad-98ce-4d214be0aa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.409785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9e2329b2-c463-4dad-98ce-4d214be0aa24 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:01.411006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9e2329b2-c463-4dad-98ce-4d214be0aa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:01.455204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:01.524197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 656/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:41:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:04.963112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97c55b-89da-4281-9a99-507e88f6baf9 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.963675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97c55b-89da-4281-9a99-507e88f6baf9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.963675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97c55b-89da-4281-9a99-507e88f6baf9 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.965293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 698/1354] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:04.975573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f338-8c83-4eaf-b577-740872187076 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.975573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f338-8c83-4eaf-b577-740872187076 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.975573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90f338-8c83-4eaf-b577-740872187076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.063664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6d90f338-8c83-4eaf-b577-740872187076 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.065610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d90f338-8c83-4eaf-b577-740872187076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.201706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 657/1355] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:05.502920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.503785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.504416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.506760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 699/1356] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 17:41:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:05.517631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdeef3-a61e-46ca-ae8a-56857f5d621f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.518858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdeef3-a61e-46ca-ae8a-56857f5d621f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.519372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdeef3-a61e-46ca-ae8a-56857f5d621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.541633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:05.605060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0fcdeef3-a61e-46ca-ae8a-56857f5d621f None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.606436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0fcdeef3-a61e-46ca-ae8a-56857f5d621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.748568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 658/1357] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:05.752430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.754023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.910244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78bab5-62e8-43f0-8e22-ef8784011816 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.989498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.030887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.031416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.031712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.031982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.033556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.126570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.128682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c0d6-b752-4aef-a541-ef77ccbf027c service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.151671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 700/1358] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:08.297926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.298456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.298456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.299846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 659/1359] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:08.305885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c789f5-f2d9-4e40-be00-962b80e8f88d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:08.306290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c789f5-f2d9-4e40-be00-962b80e8f88d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:08.306424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c789f5-f2d9-4e40-be00-962b80e8f88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:08.376934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e9c789f5-f2d9-4e40-be00-962b80e8f88d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:08.378347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e9c789f5-f2d9-4e40-be00-962b80e8f88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.420697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.499533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 701/1360] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:08.791660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:08.793166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:08.861674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:08.951334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec436f-1b0c-4cdd-ab92-bb5245bf7b3d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:09.070391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:09.071032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:09.071300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:09.071633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:09.073176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:09.097693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:09.172007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:09.173627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2c48-b20f-4705-8f35-2c06db5649f6 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:09.193338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 660/1361] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:11.469401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe0f9-e150-4a99-8dc5-38daffe1b5ff None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.469993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe0f9-e150-4a99-8dc5-38daffe1b5ff None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.469993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe0f9-e150-4a99-8dc5-38daffe1b5ff None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.472213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 702/1362] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:11.477217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c922c36-f11a-4559-9dbb-6679d4045ec8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.477694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c922c36-f11a-4559-9dbb-6679d4045ec8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.477777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c922c36-f11a-4559-9dbb-6679d4045ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.550887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6c922c36-f11a-4559-9dbb-6679d4045ec8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:11.552148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6c922c36-f11a-4559-9dbb-6679d4045ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:11.653661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:11.656977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 661/1363] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:11.952045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.952377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.952652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:11.954134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 703/1364] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 17:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:11.959448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb833-39fc-4d80-a362-9c3c648f04bd None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:11.959849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb833-39fc-4d80-a362-9c3c648f04bd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:11.960199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb833-39fc-4d80-a362-9c3c648f04bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:12.054559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c0acb833-39fc-4d80-a362-9c3c648f04bd None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:12.056400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c0acb833-39fc-4d80-a362-9c3c648f04bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.183985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 662/1365] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 17:41:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:12.191851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:12.193383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.348680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808d35-18b9-42ea-95d0-f225057bae24 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:12.369510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:12.468620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:12.469231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:12.469670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:12.470202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:12.471831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:12.566529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:12.568218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf12785-12c5-410f-a218-abef9923eec3 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:12.589502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 704/1366] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:13.139045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4754ae-9dae-452e-8bb6-768bb12ccb6d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.139638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4754ae-9dae-452e-8bb6-768bb12ccb6d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.139638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4754ae-9dae-452e-8bb6-768bb12ccb6d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:13.141327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 663/1367] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:13.146782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5225adf-424d-41f7-8e59-8a088d6cc729 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.147070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5225adf-424d-41f7-8e59-8a088d6cc729 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.147347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5225adf-424d-41f7-8e59-8a088d6cc729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:13.182010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:13.219860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a5225adf-424d-41f7-8e59-8a088d6cc729 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:13.221339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a5225adf-424d-41f7-8e59-8a088d6cc729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:13.324277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 705/1368] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:13.918773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.919025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.919306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:13.920880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 664/1369] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 24 17:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:13.925581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541431-a414-4d8f-8c4f-a25051f50230 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:13.926045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541431-a414-4d8f-8c4f-a25051f50230 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:13.926193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541431-a414-4d8f-8c4f-a25051f50230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.007451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eb541431-a414-4d8f-8c4f-a25051f50230 None None] MFA Rules not processed for user `7e690af7e92f48efb0cbc9dee2c87ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.009222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb541431-a414-4d8f-8c4f-a25051f50230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.059056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:14.102042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 706/1370] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 24 17:41:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:14.109021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:14.110721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.168727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:14.229749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a3c8b-e23f-4374-a823-1f63f6c9d133 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:14.311044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:14.311706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.311985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.312258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.313895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.425416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:14.439955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:14.441474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6c48c-9c96-4a1d-b8bf-8f4c100e048d service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:14.483575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 665/1371] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:14.631765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.632050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.632391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.634341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 707/1372] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:14.640143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb7d0-f9c3-40cf-8b44-5926358f92b0 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:14.640521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb7d0-f9c3-40cf-8b44-5926358f92b0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:14.640863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb7d0-f9c3-40cf-8b44-5926358f92b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:14.739092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8dfcb7d0-f9c3-40cf-8b44-5926358f92b0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:14.740473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8dfcb7d0-f9c3-40cf-8b44-5926358f92b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:14.843076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 666/1373] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:15.128681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:15.130213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.278801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b3398-a654-4111-ae48-c93e1eff58d8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:15.330657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:15.394665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:15.395188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:15.395456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:15.395743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:15.397321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:15.486782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:15.488446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd358a-ec55-4b84-a132-25309a89d2b9 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:15.507295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 708/1374] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:17.464626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.465411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.465411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.466553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 667/1375] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:41:17.473759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e85-1348-448b-91e2-40fa5be731fe None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:17.474182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e85-1348-448b-91e2-40fa5be731fe None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:17.474557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101e85-1348-448b-91e2-40fa5be731fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:17.568839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cc101e85-1348-448b-91e2-40fa5be731fe None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:17.570683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cc101e85-1348-448b-91e2-40fa5be731fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:17.577354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:17.668750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 709/1376] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:17.965783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:17.967455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:18.058297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:18.150948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a7cd3-1de8-4b9f-b37e-3b5e928170d0 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:18.272069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:18.272668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:18.273476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:18.273741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:18.275313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:18.295149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:18.370169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:18.371813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b816a-7ad4-44a2-9f78-fe87d16f2d67 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:18.390752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 668/1377] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:20.345161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.345664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.345766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.347039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 710/1378] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:20.352725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081d4c-f56f-4731-90c2-f0ea537e2072 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:20.353203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081d4c-f56f-4731-90c2-f0ea537e2072 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:20.353322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67081d4c-f56f-4731-90c2-f0ea537e2072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:20.425771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-67081d4c-f56f-4731-90c2-f0ea537e2072 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:20.426994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-67081d4c-f56f-4731-90c2-f0ea537e2072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:20.534830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:20.539531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 669/1379] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:20.831604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:20.833207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:20.982962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:20.996613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef027dbd-495d-4755-b39b-a5be7b9afe39 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:21.120832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:21.121340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:21.121619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:21.121877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:21.123463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:21.218507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:21.220417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d57bd-4d42-4918-bc4a-67e299c46b21 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:21.231167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:21.240669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 711/1380] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:23.184894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:23.185558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:23.185558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:23.186961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 670/1381] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:23.192220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de054-d804-4d6d-a8f1-a6ff7dba05f6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:23.192743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de054-d804-4d6d-a8f1-a6ff7dba05f6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:23.192743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de054-d804-4d6d-a8f1-a6ff7dba05f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:23.263347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5a6de054-d804-4d6d-a8f1-a6ff7dba05f6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:23.264918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5a6de054-d804-4d6d-a8f1-a6ff7dba05f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:23.378854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 712/1382] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:23.745243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:23.746833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:23.920072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3cc38-ce4d-4d22-844d-3bc2efda957b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:24.019639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:24.085730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:24.086552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:24.087038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:24.087461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:24.089495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:24.218865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:24.221260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff61c1-2a9e-48e8-81ce-78868fcbccb2 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:24.244887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 671/1383] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:26.208734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.209367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.209367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.211280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 713/1384] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:26.216559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6fa68-564b-4fdc-8d74-1b2a722f3036 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.217094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6fa68-564b-4fdc-8d74-1b2a722f3036 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.217207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6fa68-564b-4fdc-8d74-1b2a722f3036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:26.294021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-37f6fa68-564b-4fdc-8d74-1b2a722f3036 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:26.295651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-37f6fa68-564b-4fdc-8d74-1b2a722f3036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.337071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.407531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 672/1385] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:26.702029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:26.704006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:26.795324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:26.870456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3ae6-8f37-465c-8fa1-273f607ed3be None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:26.996425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:26.997433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:26.997780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:26.998133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:27.000403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:27.043461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:27.104438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:27.106012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d2000-7a4c-4203-b678-3cb8492f2712 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:27.125230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 714/1386] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:29.089309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed5f81-9370-4a17-9fbd-48e13aaf0172 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.089930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed5f81-9370-4a17-9fbd-48e13aaf0172 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.089930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed5f81-9370-4a17-9fbd-48e13aaf0172 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.091355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 673/1387] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:29.096855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.097282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.097443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.165329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:29.166557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.267105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 715/1388] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:29.550180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f5bad-04dc-4b6b-855e-c978ed83497a None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.550495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f5bad-04dc-4b6b-855e-c978ed83497a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.550772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f5bad-04dc-4b6b-855e-c978ed83497a None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.552196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 674/1389] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:29.554095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:29.555344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.557868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d5e4-c7b7-428f-8ea6-f602e567a1f2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.558148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d5e4-c7b7-428f-8ea6-f602e567a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.558416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d5e4-c7b7-428f-8ea6-f602e567a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.571492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.624998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1388d5e4-c7b7-428f-8ea6-f602e567a1f2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:29.626228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1388d5e4-c7b7-428f-8ea6-f602e567a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.676794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0d082-d21a-40e8-a4e6-97d5dd7710b0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:29.730044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 675/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:29.791666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:29.792204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:29.792484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:29.792759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:29.794371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:29.802941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:29.883978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:29.885608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66169775-0b88-45f9-90e9-e94e2d0f740d service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:29.903898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 716/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:30.036095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.036410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.036658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.038083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 676/1392] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 24 17:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:30.043704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a5d-7e12-402f-9a29-3b7f981674da None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.043986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a5d-7e12-402f-9a29-3b7f981674da None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.044330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f33a5d-7e12-402f-9a29-3b7f981674da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.128888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-70f33a5d-7e12-402f-9a29-3b7f981674da None None] MFA Rules not processed for user `2ae693a209514a79b82bc3f9ba50dccb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:30.130103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-70f33a5d-7e12-402f-9a29-3b7f981674da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.203155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.205845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 717/1393] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 24 17:41:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:30.213313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:30.214811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.228596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.322809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b804b4-55f6-4d92-af6f-754e0d517880 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:30.396921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:30.398037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:30.398310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:30.398599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:30.400241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:30.467847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:30.509062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:30.510500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641d182-3ea5-4368-b719-7f52c052fcb3 service glance] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:30.529033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 677/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:31.284991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da355cf-06f5-4705-a0cf-ca09c52dcef2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.285468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da355cf-06f5-4705-a0cf-ca09c52dcef2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.285573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da355cf-06f5-4705-a0cf-ca09c52dcef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.354424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5da355cf-06f5-4705-a0cf-ca09c52dcef2 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.355671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5da355cf-06f5-4705-a0cf-ca09c52dcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.442419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 718/1395] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:31.489177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.489653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.489885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.492176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 678/1396] 10.222.0.27 () {54 vars in 943 bytes} [Wed Aug 24 17:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:31.498730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a492dd8-edfe-40ba-aa63-cdc2476bde9f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.499014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a492dd8-edfe-40ba-aa63-cdc2476bde9f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.499287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a492dd8-edfe-40ba-aa63-cdc2476bde9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.577415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2a492dd8-edfe-40ba-aa63-cdc2476bde9f None None] MFA Rules not processed for user `ee2ab73d700247e5b5bd72ee331bb4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.578884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2a492dd8-edfe-40ba-aa63-cdc2476bde9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.615271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.692392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 719/1397] 10.222.0.27 () {58 vars in 1040 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:31.702438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:31.704005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.841957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e88174-b74e-40d7-a577-74e14b0f92af None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:31.857101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761a73-eb3f-496e-80e1-c7813ab7ede8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.857397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761a73-eb3f-496e-80e1-c7813ab7ede8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.857657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b761a73-eb3f-496e-80e1-c7813ab7ede8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.859068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 720/1398] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:41:31.864188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.864472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.864747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.881091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:31.932459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:31.933692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:31.949139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:31.949742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:31.950014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:31.950356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:31.951900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.033241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 721/1399] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:32.037997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.039637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb2754-1c80-4140-be98-655c914823a0 service swift] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.047855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 679/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:41:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:41:32.345950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:32.347353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.350879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:32.477300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fe5f0-9893-4ae0-8a84-efd665936137 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:32.622315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:32.626103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:32.627094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:32.630795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:32.630795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:32.630795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:32.739633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:32.741411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67358e1-c6ad-46cd-9d03-3d5496628143 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:32.762195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 722/1401] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:34.716346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.716936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.716936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.718379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 680/1402] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:34.723858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1261ce-4f0c-43ec-b17a-855ed6d2233f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:34.724303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1261ce-4f0c-43ec-b17a-855ed6d2233f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:34.724401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1261ce-4f0c-43ec-b17a-855ed6d2233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:34.798657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-aa1261ce-4f0c-43ec-b17a-855ed6d2233f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:34.799853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-aa1261ce-4f0c-43ec-b17a-855ed6d2233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:34.877122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:34.908463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 723/1403] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:35.198170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:35.199800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:35.280600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:35.363041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1ff37-6b22-4a93-a380-ba52283d8fcc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:35.494842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:35.495449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:35.495753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:35.496021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:35.497607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:35.533768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:35.592803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:35.595127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93df58-0663-44e5-b425-fbd0a0c53b36 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:35.616595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 681/1404] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:37.601760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:37.602312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:37.602312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:37.607668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 724/1405] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:41:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:37.609221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da713-b62e-4531-84b1-00ff3ba74127 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:37.610031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da713-b62e-4531-84b1-00ff3ba74127 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:37.612506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da713-b62e-4531-84b1-00ff3ba74127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:37.697209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-669da713-b62e-4531-84b1-00ff3ba74127 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:37.698536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-669da713-b62e-4531-84b1-00ff3ba74127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:37.809416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 682/1406] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:38.110961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:38.112568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.324128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea248b-4a1c-4185-a1ab-a63f8e9dbbf8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:38.387317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:38.452164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:38.452880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:38.453155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:38.453418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:38.455016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:38.555669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:38.557303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ceb377-41d9-4b7d-a313-839d8a22e150 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:38.576557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 725/1407] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:47.807211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:48.576857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=73 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:54.247262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799fd9-5006-4b47-83dd-2de21d9b5322 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:54.247879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799fd9-5006-4b47-83dd-2de21d9b5322 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:54.248003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19799fd9-5006-4b47-83dd-2de21d9b5322 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:54.249401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 683/1408] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 24 17:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:54.255040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9518f-6cf0-4153-ae33-1d535cf1ee4f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:54.255596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9518f-6cf0-4153-ae33-1d535cf1ee4f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:54.255698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9518f-6cf0-4153-ae33-1d535cf1ee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:54.339656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6dd9518f-6cf0-4153-ae33-1d535cf1ee4f None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:54.340916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6dd9518f-6cf0-4153-ae33-1d535cf1ee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:54.449395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 726/1409] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 24 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:54.464107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdede9-98a5-40d9-a4be-fbc211e6b986 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:54.464386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdede9-98a5-40d9-a4be-fbc211e6b986 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:54.464652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdede9-98a5-40d9-a4be-fbc211e6b986 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:54.466177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 684/1410] 10.222.0.27 () {54 vars in 918 bytes} [Wed Aug 24 17:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:54.471194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf47a8-543f-48b8-807e-eb6782e1c71d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:54.471503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf47a8-543f-48b8-807e-eb6782e1c71d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:54.471794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf47a8-543f-48b8-807e-eb6782e1c71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:54.545871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:54.549188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-dccf47a8-543f-48b8-807e-eb6782e1c71d None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:54.551076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dccf47a8-543f-48b8-807e-eb6782e1c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:54.684075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 727/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Aug 24 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:57.100220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.100781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.100781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.102119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 685/1412] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:57.107505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972ded1-0293-434d-82b3-b30136d45243 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.107899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972ded1-0293-434d-82b3-b30136d45243 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.108077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972ded1-0293-434d-82b3-b30136d45243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.173078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5972ded1-0293-434d-82b3-b30136d45243 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:57.174211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5972ded1-0293-434d-82b3-b30136d45243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.232012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:57.274365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 728/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:57.565403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:57.567545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.713541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ac278-1409-4dd0-8538-e305da5becc1 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:57.786149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:57.826040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:57.826649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:57.826928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:57.827200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:57.828914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:57.916822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:57.918450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ad60d-6570-42be-8e85-e655db790f83 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:57.936712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 686/1414] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:58.822737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:58.823265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:58.823372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:58.824761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 729/1415] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:41:58.829791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939bfe-fd25-41cd-921d-338eaff87a99 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:58.830222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939bfe-fd25-41cd-921d-338eaff87a99 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:58.830358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939bfe-fd25-41cd-921d-338eaff87a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:58.906800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c7939bfe-fd25-41cd-921d-338eaff87a99 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:58.908029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c7939bfe-fd25-41cd-921d-338eaff87a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:58.934550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:59.004495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 687/1416] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:59.155161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:59.156640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:59.299541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf4c29-f06a-4cb5-803d-82f940f93f16 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:59.340178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:59.414238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:59.414775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:59.415053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:59.415328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:59.416970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:59.504596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:59.506172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12de6db-a5e0-4cc1-be1d-76148d4f10ea service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:59.524387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 730/1417] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:00.412908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:00.414941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:00.414941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:00.414941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 688/1418] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:00.420026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40549e1-a607-4d2c-b2d3-b304fe20f51f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:00.420484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40549e1-a607-4d2c-b2d3-b304fe20f51f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:00.420590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40549e1-a607-4d2c-b2d3-b304fe20f51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:00.474449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:00.496267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f40549e1-a607-4d2c-b2d3-b304fe20f51f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:00.497478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f40549e1-a607-4d2c-b2d3-b304fe20f51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:00.595231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 731/1419] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:00.749459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:00.751121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:00.898614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1833f-67c1-4273-ada7-76837184e293 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:00.911291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:01.011135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:01.011684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:01.011965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:01.012237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:01.013825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:01.101133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:01.102767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbd84c-ef96-4f5b-96d8-09ce9955dd09 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:01.120991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 689/1420] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:02.002761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.003330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.003330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.005266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 732/1421] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:02.010145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7297c36-db54-4d39-ba5a-b2a56c555350 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.010551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7297c36-db54-4d39-ba5a-b2a56c555350 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.010692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7297c36-db54-4d39-ba5a-b2a56c555350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.032499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.088798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c7297c36-db54-4d39-ba5a-b2a56c555350 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:02.090016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c7297c36-db54-4d39-ba5a-b2a56c555350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.190821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 690/1422] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:02.343110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.344587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.455087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.492524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f26e4c-5f5f-46a3-a651-f4a051e33e8b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.605700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.606253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.606535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.606805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.608438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:02.684114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:02.698340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.699982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258abb8-8241-469b-ba92-a7ad6644d917 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:02.718080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 733/1423] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:03.575651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.576267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.576267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.577554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 691/1424] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:03.582560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f52cb-0bb8-4185-8f41-277ce821bed5 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.582973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f52cb-0bb8-4185-8f41-277ce821bed5 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.583083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f52cb-0bb8-4185-8f41-277ce821bed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.665098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e20f52cb-0bb8-4185-8f41-277ce821bed5 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:03.666397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e20f52cb-0bb8-4185-8f41-277ce821bed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.763115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 734/1425] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:03.913056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.914546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.998655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:04.064318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65215dcc-a58b-4346-890e-707d97dd430e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.176839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.177364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.177645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.177916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.179582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.228916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:04.270221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.271904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe5c96-cd16-4d15-b1e1-7564e20ec779 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.290071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 692/1426] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:05.090494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.091085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.091198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.092789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 735/1427] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:05.098500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3993-b1e0-4a43-9359-e3e818bfd6ba None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.098941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3993-b1e0-4a43-9359-e3e818bfd6ba None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.099046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba3993-b1e0-4a43-9359-e3e818bfd6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.183974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-70ba3993-b1e0-4a43-9359-e3e818bfd6ba None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:05.185213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-70ba3993-b1e0-4a43-9359-e3e818bfd6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.286095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 693/1428] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:05.439939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:05.441460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.490321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:05.586686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c96c9-b9cb-4842-80b9-dbbf67b16dc8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:05.699660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:05.700199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.700475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.700742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.702320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.709154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:05.790390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:05.792016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a8931-e9b8-4a95-ad18-52b9eec2cfc3 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:05.809961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 736/1429] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:06.609300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.609853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.609853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.611209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 694/1430] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:06.616164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cdef6-2f15-4e70-a7fe-bbf50dba2214 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.616613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cdef6-2f15-4e70-a7fe-bbf50dba2214 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.616714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cdef6-2f15-4e70-a7fe-bbf50dba2214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.690967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-166cdef6-2f15-4e70-a7fe-bbf50dba2214 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:06.692701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-166cdef6-2f15-4e70-a7fe-bbf50dba2214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.759198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:06.788087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 737/1431] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:06.944878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.946467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.964622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:07.094955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8113a-3cff-4a6a-9879-aca502b444fe None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:07.181945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:07.208239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:07.208852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:07.209130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:07.209402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:07.211134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:07.299201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:07.301466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136fcd-99bb-4c68-b812-def335ce9a58 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:07.319556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 695/1432] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:08.122331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:08.122912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:08.122912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:08.124264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 738/1433] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:08.129099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aaf5e-24ac-4c9f-938c-9f8111d1c951 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:08.129545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aaf5e-24ac-4c9f-938c-9f8111d1c951 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:08.129643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aaf5e-24ac-4c9f-938c-9f8111d1c951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:08.204736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-201aaf5e-24ac-4c9f-938c-9f8111d1c951 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:08.205957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-201aaf5e-24ac-4c9f-938c-9f8111d1c951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:08.248328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:08.303369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 696/1434] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:08.460135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:08.461608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:08.606475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f3d0d-465d-4771-8822-47f81ce44f6d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:08.660737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:08.719571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:08.720184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:08.720464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:08.720737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:08.722345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:08.811501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:08.813136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610959fd-d049-4119-90b9-356f72a9a6d2 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:08.831685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 739/1435] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:09.629900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:09.630485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:09.630485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:09.631848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 697/1436] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:09.637028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18a58-ab30-48e1-93f7-5ba6acf23d0f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:09.637473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18a58-ab30-48e1-93f7-5ba6acf23d0f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:09.637578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18a58-ab30-48e1-93f7-5ba6acf23d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:09.721658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-dfb18a58-ab30-48e1-93f7-5ba6acf23d0f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:09.722934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dfb18a58-ab30-48e1-93f7-5ba6acf23d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:09.724969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:09.820046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 740/1437] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:09.969729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:09.971255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:10.120424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63bf49-aa34-4698-9831-adc4679b834c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:10.146594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:10.232883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:10.233406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:10.233685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:10.233956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:10.235583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:10.324349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:10.325976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd7bb8-b34c-4432-b141-6b0d395a0cda service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:10.343956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 698/1438] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:12.188399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.188933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.188933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.190290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 741/1439] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:12.195932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053f56-0a9c-4d6e-8839-c2d00e7eada8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.196396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053f56-0a9c-4d6e-8839-c2d00e7eada8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.196513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053f56-0a9c-4d6e-8839-c2d00e7eada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.237492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:12.262666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-41053f56-0a9c-4d6e-8839-c2d00e7eada8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:12.263935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-41053f56-0a9c-4d6e-8839-c2d00e7eada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.361306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 699/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:12.649203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:12.650772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.753236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365930a-c760-4be6-a936-fc9b2eecc4ac None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:12.785720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:12.825620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:12.826216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:12.826492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:12.826758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:12.828381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:12.934938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:12.936399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d36835-73c1-42db-9c92-6c809e37d2e2 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:12.954447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 742/1441] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:15.006793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.007354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.007354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.008733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 700/1442] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:15.013925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e648f9-d408-4a3b-a740-04280d14aa82 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.014349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e648f9-d408-4a3b-a740-04280d14aa82 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.014472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e648f9-d408-4a3b-a740-04280d14aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.080126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-61e648f9-d408-4a3b-a740-04280d14aa82 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:15.081323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-61e648f9-d408-4a3b-a740-04280d14aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.094624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:15.177514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 743/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:15.460204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:15.461754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.565070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725801c-5e44-4c7c-93a5-d51c18cac9a7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:15.577585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:15.639855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:15.640459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:15.640737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:15.641003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:15.642602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:15.756009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:15.757465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60488852-a6a3-483b-a1a3-ac28515936f3 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:15.776310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 701/1444] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:18.003379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.003970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.003970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.005912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 744/1445] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:18.011226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3844c5e-0e44-4efa-bf0f-f3d138716cce None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.011725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3844c5e-0e44-4efa-bf0f-f3d138716cce None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.011833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3844c5e-0e44-4efa-bf0f-f3d138716cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.059271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:18.076962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a3844c5e-0e44-4efa-bf0f-f3d138716cce None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:18.078175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a3844c5e-0e44-4efa-bf0f-f3d138716cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.171455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 702/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:18.456650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:18.458218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.561717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec213-06e2-4f64-b155-eb6945d4c1b1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:18.601163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:18.635750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:18.636354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:18.636631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:18.636900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:18.638487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:18.745518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:18.746958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c666f-f0aa-4859-9068-2c19818d235d service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:18.765348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 745/1447] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:20.789504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:20.790032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:20.790032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:20.791305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 703/1448] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:42:20.796756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6684-1546-4af3-b9b6-3bbeb108cb20 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:20.797148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6684-1546-4af3-b9b6-3bbeb108cb20 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:20.797309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f6684-1546-4af3-b9b6-3bbeb108cb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:20.862303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3d7f6684-1546-4af3-b9b6-3bbeb108cb20 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:20.863555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3d7f6684-1546-4af3-b9b6-3bbeb108cb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:20.884371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:20.960145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 746/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:21.242888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:21.244486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:21.348635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9b6e4-1c05-44a5-bb98-f144271f831d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:21.368580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:21.422474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:21.423074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:21.423347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:21.423644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:21.425236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:21.534769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:21.536248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8efc72-cdd9-41ba-b985-e8cf73a634ef service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:21.554252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 704/1450] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:23.599488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:23.600034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:23.600034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:23.601393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 747/1451] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:23.606598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e6ece-1e41-42c2-9eb5-12fbba1083d9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:23.607050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e6ece-1e41-42c2-9eb5-12fbba1083d9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:23.607149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e6ece-1e41-42c2-9eb5-12fbba1083d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:23.669858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:23.672599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3e5e6ece-1e41-42c2-9eb5-12fbba1083d9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:23.674654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3e5e6ece-1e41-42c2-9eb5-12fbba1083d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:23.774156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 705/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:24.052360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:24.053946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:24.157917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac12f9-990f-4aec-a055-77f3ac9b9103 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:24.206946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:24.231384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:24.232021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:24.232294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:24.232560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:24.234163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:24.342120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:24.343771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543386-7e60-42f7-8921-cfe5c936417c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:24.361996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 748/1453] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:26.307435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:26.308078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:26.308078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:26.309478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 706/1454] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:26.314724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716dc94-db34-4402-85f8-59432f760ea8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:26.315196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716dc94-db34-4402-85f8-59432f760ea8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:26.315314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716dc94-db34-4402-85f8-59432f760ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:26.381471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9716dc94-db34-4402-85f8-59432f760ea8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:26.382792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9716dc94-db34-4402-85f8-59432f760ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:26.410628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:26.479866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 749/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:26.764252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:26.765793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:26.868511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66558b9-76ac-4064-9de5-2d0600e683a7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:26.902151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:26.943533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:26.944144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:26.944424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:26.944693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:26.946306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:27.064548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:27.066015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7107fb-0552-4fd8-9ee1-0d81ad82dc99 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:27.085080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 707/1456] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:29.028130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:29.028700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:29.028700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:29.030039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 750/1457] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:29.035205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bbd8-15b1-4746-8472-b5cbe4b2649b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:29.036208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bbd8-15b1-4746-8472-b5cbe4b2649b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:29.036333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bbd8-15b1-4746-8472-b5cbe4b2649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:29.102642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e112bbd8-15b1-4746-8472-b5cbe4b2649b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:29.103901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e112bbd8-15b1-4746-8472-b5cbe4b2649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:29.111936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:29.199032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 708/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:29.482840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:29.484433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:29.590028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b13c5-0fc6-48bb-bbe6-006fe87ccb61 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:29.645835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:29.664425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:29.665029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:29.665303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:29.665572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:29.667172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:29.777349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:29.778737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd0f7-2f65-48d2-806f-40d8d443f423 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:29.796839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 751/1459] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:31.729070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:31.729586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:31.729586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:31.730966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 709/1460] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:31.736365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902fa7-2315-49fe-8fd7-9cac1d6545c4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:31.736903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902fa7-2315-49fe-8fd7-9cac1d6545c4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:31.736903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902fa7-2315-49fe-8fd7-9cac1d6545c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:31.811280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e0902fa7-2315-49fe-8fd7-9cac1d6545c4 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:31.812649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e0902fa7-2315-49fe-8fd7-9cac1d6545c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:31.850997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:31.907722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 752/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:32.188425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:32.190032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:32.293842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104c6bc-8432-489e-82c7-d528bc06788c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:32.331181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:32.367911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:32.368520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:32.368798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:32.369074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:32.370673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:32.483094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:32.484562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69ddfd-efa1-47ed-8486-e8480fe7abff service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:32.506240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 710/1462] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:34.484288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:34.484975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:34.484975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:34.486192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 753/1463] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:34.492321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5e59-64b5-48ee-8afd-24155a86ab0c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:34.492761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5e59-64b5-48ee-8afd-24155a86ab0c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:34.492870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5e59-64b5-48ee-8afd-24155a86ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:34.559455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-edcd5e59-64b5-48ee-8afd-24155a86ab0c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:34.560697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-edcd5e59-64b5-48ee-8afd-24155a86ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:34.579208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:34.660952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 711/1464] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:42:34.940299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:34.941834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:35.045125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a0473-3f31-4d67-aa86-ac1efda60097 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:35.110858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:35.117875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:35.118511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:35.118789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:35.119056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:35.120660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:35.228332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:35.229765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eaa698-867e-4071-9471-2268c7c1809e service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:35.247448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 754/1465] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:37.172077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:37.172623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:37.172623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:37.173936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 712/1466] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:37.179455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1673b-4654-4d94-a91b-49ffe5f163ff None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:37.179877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1673b-4654-4d94-a91b-49ffe5f163ff None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:37.180035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1673b-4654-4d94-a91b-49ffe5f163ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:37.245567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b4e1673b-4654-4d94-a91b-49ffe5f163ff None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:37.246789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b4e1673b-4654-4d94-a91b-49ffe5f163ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:37.295258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:37.344745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 755/1467] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:37.635143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:37.636823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:37.739651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d0c9-80ff-43cc-a36b-321271fb8f9f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:37.786756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:37.821585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:37.822185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:37.822457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:37.822720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:37.824334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:37.930623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:37.932079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b7397-3e6a-4bbf-81af-088fb6ba4227 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:37.950511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 713/1468] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:47.342719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=22 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:47.948345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=66 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:57.496760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a9d4-9dae-4600-ae3d-fa6d446f53e0 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:57.497300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a9d4-9dae-4600-ae3d-fa6d446f53e0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:57.497300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a9d4-9dae-4600-ae3d-fa6d446f53e0 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:57.498662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 756/1469] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:57.504132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ce4a-715e-412a-b0b3-537d34fbb4ab None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:57.504552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ce4a-715e-412a-b0b3-537d34fbb4ab None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:57.504687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31ce4a-715e-412a-b0b3-537d34fbb4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:57.570014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ed31ce4a-715e-412a-b0b3-537d34fbb4ab None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:57.571227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ed31ce4a-715e-412a-b0b3-537d34fbb4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:57.666151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 714/1470] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:57.965117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:57.965405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:57.965681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:57.967010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 757/1471] 10.222.0.27 () {54 vars in 967 bytes} [Wed Aug 24 17:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:42:57.972188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cf0e-f655-45b5-8b21-aa4b06d6f359 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:57.972499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cf0e-f655-45b5-8b21-aa4b06d6f359 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:57.972774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cf0e-f655-45b5-8b21-aa4b06d6f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:58.170341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9206cf0e-f655-45b5-8b21-aa4b06d6f359 None None] MFA Rules not processed for user `2ae693a209514a79b82bc3f9ba50dccb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:58.171613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9206cf0e-f655-45b5-8b21-aa4b06d6f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:58.192441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:58.249901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 715/1472] 10.222.0.27 () {58 vars in 1064 bytes} [Wed Aug 24 17:42:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:58.258247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:58.259813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:58.362295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d03fc-f6bf-490d-b0d9-318da844bbce None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:58.434307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:58.434909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:58.435182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:58.435447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:58.437065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:58.490198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:58.545396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:58.546829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2ff2d-88b9-4d33-a8b7-e1c8ad76bc3b service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:58.566350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 758/1473] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:43:00.527902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.528366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.528485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.529962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 716/1474] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:00.535020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b7896-6fc5-4355-83e3-5a1a4d779325 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:00.535436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b7896-6fc5-4355-83e3-5a1a4d779325 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:00.535595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b7896-6fc5-4355-83e3-5a1a4d779325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:00.599915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-334b7896-6fc5-4355-83e3-5a1a4d779325 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:00.601645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-334b7896-6fc5-4355-83e3-5a1a4d779325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:00.695674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 759/1475] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:00.968951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:00.970773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:01.099783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce702f-c8b4-4819-a5e9-e6d6b0678b33 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:01.149017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:01.193647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:01.194870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:01.195040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:01.195306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:01.195965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:01.197704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:43:01.197704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:43:01.197704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:43:01.198841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08039f-a7b3-4c26-8c92-d7f2d53a930e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:01.217973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 717/1476] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Aug 24 17:43:00 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:43:03.111785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:03.112356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:03.112356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:03.113660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 760/1477] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:03.119054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010373eb-5d6a-4f05-a77e-2f0fb640454d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:03.119504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010373eb-5d6a-4f05-a77e-2f0fb640454d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:03.119684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010373eb-5d6a-4f05-a77e-2f0fb640454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:03.185001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-010373eb-5d6a-4f05-a77e-2f0fb640454d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:03.186232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-010373eb-5d6a-4f05-a77e-2f0fb640454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:03.283355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 718/1478] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:43:03.577274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:03.578829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:03.589319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:03.725140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d127b9f-0f94-4f97-80aa-0d6a3ccdbeec None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:03.804592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:03.838212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:03.838818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:03.839099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:03.839373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:03.841022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:03.932447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:03.934042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8608a-f2e0-406d-88f4-2622acf9bde4 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:03.952179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 761/1479] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:06.056004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:06.056613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:06.056613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:06.057934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 719/1480] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:06.063397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b08d3-2097-4429-8b80-d4c023f59cbb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:06.063885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b08d3-2097-4429-8b80-d4c023f59cbb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:06.063994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b08d3-2097-4429-8b80-d4c023f59cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:06.132672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-992b08d3-2097-4429-8b80-d4c023f59cbb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:06.133985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-992b08d3-2097-4429-8b80-d4c023f59cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:06.179750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:06.230214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 762/1481] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:06.514553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:06.516139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:06.539982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:06.618479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8cc94-1ce0-4797-a450-a710e424a9d9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:06.691658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:06.692266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:06.692546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:06.692815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:06.694412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:06.766742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:06.801392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:06.802834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586619b-f439-4761-a7cb-e4faba5a5cf9 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:06.821364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 720/1482] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:08.790133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:08.790621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:08.790621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:08.791993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 763/1483] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:08.797149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aacc-82a2-443d-9837-849b6699a249 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:08.797582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aacc-82a2-443d-9837-849b6699a249 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:08.797707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aacc-82a2-443d-9837-849b6699a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:08.862961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3cb2aacc-82a2-443d-9837-849b6699a249 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:08.864198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3cb2aacc-82a2-443d-9837-849b6699a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:08.959496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 721/1484] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:09.245818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:09.247391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:09.350021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1ed9b-bab3-4fdf-be18-94d8e5d7bb6a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:09.425127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:09.426874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:09.427768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:09.428574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:09.429077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:09.430913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:09.539778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:09.541219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dbe28-2346-4dfe-848a-b0f31ff3f5ce service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:09.558787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 764/1485] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:11.541060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:11.541627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:11.541627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:11.542938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 722/1486] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:11.548454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896218-3324-4ff5-aadd-8f802ad7dcc2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:11.548902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896218-3324-4ff5-aadd-8f802ad7dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:11.549008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee896218-3324-4ff5-aadd-8f802ad7dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:11.621323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ee896218-3324-4ff5-aadd-8f802ad7dcc2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:11.622543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ee896218-3324-4ff5-aadd-8f802ad7dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:11.676485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:11.720809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 765/1487] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:12.018581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:12.020255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:12.058126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:12.126338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37888f12-ced6-49af-a21d-cc90f2cc4e28 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:12.197264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:12.197867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:12.198143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:12.198409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:12.200021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:12.279623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:12.308216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:12.309658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081aa0ba-4a46-4c89-93b0-272a8dbf198c service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:12.326942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 723/1488] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:14.379880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8c5b-f27a-46b8-a84f-0a7dc7748e97 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:14.380449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8c5b-f27a-46b8-a84f-0a7dc7748e97 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:14.380449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f8c5b-f27a-46b8-a84f-0a7dc7748e97 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:14.381791 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 766/1489] 10.222.0.27 () {54 vars in 927 bytes} [Wed Aug 24 17:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:14.387719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e616c-9ffd-47db-af6a-c398817578fb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:14.388151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e616c-9ffd-47db-af6a-c398817578fb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:14.388262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e616c-9ffd-47db-af6a-c398817578fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:14.452434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-554e616c-9ffd-47db-af6a-c398817578fb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:14.453657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-554e616c-9ffd-47db-af6a-c398817578fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:14.547975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 724/1490] 10.222.0.27 () {58 vars in 1024 bytes} [Wed Aug 24 17:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:14.850139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:14.850419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:14.850689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:14.852177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 767/1491] 10.222.0.27 () {54 vars in 965 bytes} [Wed Aug 24 17:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:43:14.857906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e707b2-6f42-413e-9c99-768dcd3a97bc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:14.858192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e707b2-6f42-413e-9c99-768dcd3a97bc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:14.858468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e707b2-6f42-413e-9c99-768dcd3a97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:14.910368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:14.936011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a4e707b2-6f42-413e-9c99-768dcd3a97bc None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:43:14.937217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a4e707b2-6f42-413e-9c99-768dcd3a97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:15.011792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 725/1492] 10.222.0.27 () {58 vars in 1062 bytes} [Wed Aug 24 17:43:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:43:15.023566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:15.025568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:15.126657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d29a-d051-445b-b40b-a193c7ea3c58 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:15.197479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:15.198077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:15.198349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:15.198615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:15.200215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:15.209033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:15.308425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:15.309864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b9874-9dec-4125-b219-5c4193b4cc90 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:15.328547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 768/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:25.009874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=41 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:25.326247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=50 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:28.377442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:28.378040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:28.378040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:28.452998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:28.454222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:28.550446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 726/1494] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:28.559332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:28.559871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:28.559980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:28.635555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:28.636691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:28.732678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 769/1495] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:28.737483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:28.738677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:28.750727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08474dd2-20fb-43b7-9b96-0379aeb62bd8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:28.810862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:28.845097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:28.845702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:28.845979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:28.846249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:28.847138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:44:28.847434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:28.848474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:28.848474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:44:28.848474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:44:28.849359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:28.860301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 727/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:44:28.869123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:28.878319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8850802-f130-4f50-8c3d-592f127e34c0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:28.939621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:28.972816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:28.973418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:28.973697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:28.973964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:28.974579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:28.976881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:28.976881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:44:28.976881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:44:28.978088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:29.033795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 770/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:44:28 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:44:29.038121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:29.040169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:29.073837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbe2ef-e7a4-4fc4-be0a-02176631f310 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:29.167584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:29.168191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.168471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.168746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.169372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:29.170386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:29.170386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:44:29.170386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:44:29.171629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:29.213399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 728/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:44:29 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:44:29.217420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:29.218960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:29.280529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def0386-9a00-4582-8db7-ec949882b40b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:29.363931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:29.375356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:29.375989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.376268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.376539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.377172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:29.378194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:29.378194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:44:29.378194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:44:29.379077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:29.391320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 771/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:44:29.395372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:29.396973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:29.408771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae928149-7c9b-4289-9f1e-5d8153a6c934 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:29.434006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:29.503239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:29.503903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.504181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.504450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.505091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:29.506098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:29.506098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:44:29.506098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:44:29.507022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3850f-c0d0-4cc6-8039-efab4ec94d19 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:29.519603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 729/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:44:29.524159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:29.525711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:29.561637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d00b-b31c-4407-b3af-7d70372ab1e8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:29.655364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:29.656053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.656330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.656601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] PATH_INFO: `/v3/projects/ffca6484c0fa42fb9cfea11e8c887eef/users/89b4df5f559f42c1a561179dab448d46/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.678557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffca6484c0fa42fb9cfea11e8c887eef, user_id=89b4df5f559f42c1a561179dab448d46, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:29.679698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:29.679698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:44:29.679698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:44:29.681685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691ba03-81a4-4c87-a137-e3dcbdaba6f0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:29.706071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 772/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:44:29 2022] PUT /identity/v3/projects/ffca6484c0fa42fb9cfea11e8c887eef/users/89b4df5f559f42c1a561179dab448d46/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:29.718719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783be2-bd02-45df-9064-44c3ef40c6ce None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.719004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783be2-bd02-45df-9064-44c3ef40c6ce None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.719276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a783be2-bd02-45df-9064-44c3ef40c6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.775748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8a783be2-bd02-45df-9064-44c3ef40c6ce None None] MFA Rules not processed for user `89b4df5f559f42c1a561179dab448d46`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:29.776969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8a783be2-bd02-45df-9064-44c3ef40c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:29.853661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 730/1502] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:44:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:29.866911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.867197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.867472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.868842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 773/1503] 10.222.0.27 () {54 vars in 964 bytes} [Wed Aug 24 17:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:29.873772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd938141-6500-4574-bbd7-0d3bcb418c84 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:29.874053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd938141-6500-4574-bbd7-0d3bcb418c84 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:29.874324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd938141-6500-4574-bbd7-0d3bcb418c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:29.905950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:29.949990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-dd938141-6500-4574-bbd7-0d3bcb418c84 None None] MFA Rules not processed for user `7e690af7e92f48efb0cbc9dee2c87ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:29.951186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dd938141-6500-4574-bbd7-0d3bcb418c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:30.024701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 731/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Aug 24 17:44:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:30.032450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:30.033969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:30.085369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:30.140884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdeb12-fe49-4acd-9f9b-4027f32f5b3b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:30.212170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:30.212774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:30.213053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:30.213323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:30.214927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:30.313158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:30.314533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:30.318050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:30.333526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 774/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:30.379261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9edda2-86d1-4c4c-b5d5-fccdf0f9137c None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:30.379576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9edda2-86d1-4c4c-b5d5-fccdf0f9137c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:30.379877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9edda2-86d1-4c4c-b5d5-fccdf0f9137c None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:30.381148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 732/1506] 10.222.0.27 () {52 vars in 830 bytes} [Wed Aug 24 17:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:44:30.385452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:30.386675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:30.484676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67001ba-166b-450b-90c0-0c2381f61273 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:30.577629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:30.578307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:30.578443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:30.578710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] PATH_INFO: `/v3/users/89b4df5f559f42c1a561179dab448d46` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:30.590602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b4df5f559f42c1a561179dab448d46)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:30.591654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:30.591654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:44:30.591654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:44:30.592875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:30.603079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:30.658308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b4df5f559f42c1a561179dab448d46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:30.658759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b4df5f559f42c1a561179dab448d46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:30.670303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b4df5f559f42c1a561179dab448d46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:30.684266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b4df5f559f42c1a561179dab448d46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:30.825877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b4df5f559f42c1a561179dab448d46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:30.831743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 775/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:44:30 2022] DELETE /identity/v3/users/89b4df5f559f42c1a561179dab448d46 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:30.836545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:30.837009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:30.837102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:30.900173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:44:30.901416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:30.978177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:30.991255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 733/1508] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:44:31.001575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:31.002851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:31.101031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de61f54-3bd7-4d91-9435-3b1d39feea4c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:31.156625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:31.179088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:31.179732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:31.180014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:31.180286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:31.181892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:31.289175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:31.290665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e055-5252-4050-9177-0264c9b9b773 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:31.308159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 776/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:44:31.658565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:44:31.659894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:44:31.672020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73e8f7-b1f3-49a9-b401-c239372ae9c5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:44:31.767762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:44:31.768446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:44:31.768720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:44:31.769016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] PATH_INFO: `/v3/projects/ffca6484c0fa42fb9cfea11e8c887eef` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:44:31.779261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffca6484c0fa42fb9cfea11e8c887eef)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:44:31.780401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:44:31.780401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:44:31.780401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:44:31.781603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:44:31.838993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffca6484c0fa42fb9cfea11e8c887eef', 'request_id': 'req-018213dd-475a-489f-a2b4-b983e8c279ff'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:31.845061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffca6484c0fa42fb9cfea11e8c887eef', 'request_id': 'req-018213dd-475a-489f-a2b4-b983e8c279ff'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:31.856076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffca6484c0fa42fb9cfea11e8c887eef', 'request_id': 'req-018213dd-475a-489f-a2b4-b983e8c279ff'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:31.885725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] The token cache is being invalidate because project ffca6484c0fa42fb9cfea11e8c887eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:44:31.886091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:31.886526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-018213dd-475a-489f-a2b4-b983e8c279ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:44:31.887965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 734/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:44:31 2022] DELETE /identity/v3/projects/ffca6484c0fa42fb9cfea11e8c887eef => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:41.305929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=62 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:41.883422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:59.122059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:59.122733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:59.122733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:59.124079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 777/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:45:59.128789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa291-251c-454a-9de1-c82b84024d5c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:59.129246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa291-251c-454a-9de1-c82b84024d5c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:59.129356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa291-251c-454a-9de1-c82b84024d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:59.208887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e7baa291-251c-454a-9de1-c82b84024d5c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:45:59.210091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e7baa291-251c-454a-9de1-c82b84024d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:59.310357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 735/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:45:59.462800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:59.465101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:59.583981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c3f57-b5b5-48ef-8280-6905c89eba21 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:59.589842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:59.590438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:59.590715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:59.590980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:59.599213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:59.600833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5119fe0-e4f1-47a7-afbc-b37b1db6879a service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:59.611180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 778/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:00.441954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98018e-ce56-4dbe-bdb9-dea504500fab None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:00.442522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98018e-ce56-4dbe-bdb9-dea504500fab None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:00.442522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98018e-ce56-4dbe-bdb9-dea504500fab None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:00.443907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 736/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:00.448681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4536df-21b2-4687-ad1f-ed360aa169d8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:00.449077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4536df-21b2-4687-ad1f-ed360aa169d8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:00.449230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4536df-21b2-4687-ad1f-ed360aa169d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:00.468283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ee4536df-21b2-4687-ad1f-ed360aa169d8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:00.469492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ee4536df-21b2-4687-ad1f-ed360aa169d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:00.478004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 779/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:00.640973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98018e-ce56-4dbe-bdb9-dea504500fab None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:00.655164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98018e-ce56-4dbe-bdb9-dea504500fab None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:00.660673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:00.661267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:00.661553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:00.661805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:00.670486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:00.672112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cdebc6-6c90-423a-958d-550a408a55c6 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:00.682029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 737/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:01.161689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1666190-48e3-4072-aa11-17561749ed54 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:01.161975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1666190-48e3-4072-aa11-17561749ed54 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:01.162272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1666190-48e3-4072-aa11-17561749ed54 None None] PATH_INFO: `/v3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:01.165010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 780/1517] 10.222.0.62 () {54 vars in 925 bytes} [Wed Aug 24 17:46:01 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:46:01.172914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766984-5e74-49ce-9b95-127f389d16b0 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:01.173206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766984-5e74-49ce-9b95-127f389d16b0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:01.173479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766984-5e74-49ce-9b95-127f389d16b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:01.231009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7a766984-5e74-49ce-9b95-127f389d16b0 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:01.232247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7a766984-5e74-49ce-9b95-127f389d16b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:01.248741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:01.302782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 738/1518] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Aug 24 17:46:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:05.048690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a5b0-b1e3-433b-b307-b531204aa232 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:05.049190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a5b0-b1e3-433b-b307-b531204aa232 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:05.049190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167a5b0-b1e3-433b-b307-b531204aa232 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:05.050491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 781/1519] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 24 17:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:05.055785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f7de-3721-4248-bcf8-e1568ddd9360 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:05.056208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f7de-3721-4248-bcf8-e1568ddd9360 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:05.056372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f7de-3721-4248-bcf8-e1568ddd9360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:05.076189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7961f7de-3721-4248-bcf8-e1568ddd9360 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:05.077452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7961f7de-3721-4248-bcf8-e1568ddd9360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:05.086167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 739/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 24 17:46:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:05.093581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167a5b0-b1e3-433b-b307-b531204aa232 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:05.107544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167a5b0-b1e3-433b-b307-b531204aa232 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:05.112503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:05.113095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:05.113373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:05.113670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:05.122478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:05.124089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f7f20-645f-4771-be19-012e62cb0c32 service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:05.133654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 782/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:11.856209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77daf-ccb1-40c2-888f-f1be06e7e7ba None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:11.856685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77daf-ccb1-40c2-888f-f1be06e7e7ba None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:11.856781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77daf-ccb1-40c2-888f-f1be06e7e7ba None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:11.858121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 740/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:11.862765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe419a-a2ce-4528-897a-d09294889380 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:11.863187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe419a-a2ce-4528-897a-d09294889380 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:11.863307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe419a-a2ce-4528-897a-d09294889380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:11.882533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3bbe419a-a2ce-4528-897a-d09294889380 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:11.883742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3bbe419a-a2ce-4528-897a-d09294889380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:11.891558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 783/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:12.047941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f77daf-ccb1-40c2-888f-f1be06e7e7ba None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:12.062452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f77daf-ccb1-40c2-888f-f1be06e7e7ba None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:12.068208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:12.068807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:12.069117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:12.069393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:12.077752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:12.079441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d18e-1c72-4792-9f75-9b6ce967e73b service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:12.089602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 741/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:12.899662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbded2-7e37-4184-9505-619edb3865a9 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:12.900213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbded2-7e37-4184-9505-619edb3865a9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:12.900213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbded2-7e37-4184-9505-619edb3865a9 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:12.901455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 784/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:12.906166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ad87f-d8dd-4e12-abdb-155da5cba2e2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:12.906582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ad87f-d8dd-4e12-abdb-155da5cba2e2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:12.906715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ad87f-d8dd-4e12-abdb-155da5cba2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:12.925949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a59ad87f-d8dd-4e12-abdb-155da5cba2e2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:12.927140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a59ad87f-d8dd-4e12-abdb-155da5cba2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:12.935800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 742/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:13.085871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbded2-7e37-4184-9505-619edb3865a9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:13.100325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbded2-7e37-4184-9505-619edb3865a9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:13.105738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:13.106321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:13.106598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:13.106863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:13.114650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:13.116257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d33838-f939-4374-b1a3-dc208a97cf22 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:13.125909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 785/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:16.896347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:16.896873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:16.896873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:16.898221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 743/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:16.903502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32998-dc50-4745-8586-50a474534a37 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:16.903917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32998-dc50-4745-8586-50a474534a37 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:16.904087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c32998-dc50-4745-8586-50a474534a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:16.923793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-08c32998-dc50-4745-8586-50a474534a37 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:16.925026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-08c32998-dc50-4745-8586-50a474534a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:16.933575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 786/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:17.226738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:17.228545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:17.306673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c0d8-de1d-4e90-8bec-aa5ff6945448 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:17.310236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:17.310839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:17.311160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:17.311624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:17.321822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:17.323263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e6fef-7501-4015-97f3-ec0fd0b431ac service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:17.332645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 744/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:19.243007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabbe5-0063-4d43-b30c-704af97fd850 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:19.243599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabbe5-0063-4d43-b30c-704af97fd850 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:19.243599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabbe5-0063-4d43-b30c-704af97fd850 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:19.244934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 787/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:19.250210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695503-9289-4f36-8c33-316c7650e203 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:19.250612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695503-9289-4f36-8c33-316c7650e203 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:19.250760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06695503-9289-4f36-8c33-316c7650e203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:19.270335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-06695503-9289-4f36-8c33-316c7650e203 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:19.271562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-06695503-9289-4f36-8c33-316c7650e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:19.279880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 745/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:19.563188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cabbe5-0063-4d43-b30c-704af97fd850 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:19.574431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cabbe5-0063-4d43-b30c-704af97fd850 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:19.577950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:19.578544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:19.578819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:19.579087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:19.588456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:19.589878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d911b4-df18-47c5-ad6d-9ccab5426c9c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:19.599386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 788/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:23.032838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:23.033446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:23.033446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:23.034723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 746/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:23.039853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86a13-50a8-430c-a142-6dcd0e3347c6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:23.040821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86a13-50a8-430c-a142-6dcd0e3347c6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:23.040926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec86a13-50a8-430c-a142-6dcd0e3347c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:23.048382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:23.061458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1ec86a13-50a8-430c-a142-6dcd0e3347c6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:23.062658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1ec86a13-50a8-430c-a142-6dcd0e3347c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:23.070390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 789/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:23.615201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:23.616909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:23.682259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:23.695977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419b9e6-b33d-4a6a-8061-c00192708a6a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:23.699552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:23.700162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:23.700442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:23.700714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:23.709666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:23.711140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd28bf-0353-43e0-8766-37925bf63d8e service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:23.720515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 747/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:25.650168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:25.650718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:25.650718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:25.652106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 790/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:46:25.657304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7f82d-effa-476b-8700-2a6b203c2aac None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:25.657734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7f82d-effa-476b-8700-2a6b203c2aac None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:25.657833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7f82d-effa-476b-8700-2a6b203c2aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:25.677730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a7b7f82d-effa-476b-8700-2a6b203c2aac None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:46:25.678934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a7b7f82d-effa-476b-8700-2a6b203c2aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:25.687579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 748/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:46:26.234167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:46:26.235891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:46:26.248649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b1a71-150c-497b-aa37-13f92379d549 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:46:26.252159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:46:26.252738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:46:26.253009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:46:26.253277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:46:26.261889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:46:26.263284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca1ccc-f266-487b-8437-aa7d3c4bde1f service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:46:26.272429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 791/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:46:35.671761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=11 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:46:36.268411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=8 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:49.087673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6c6-598c-49f9-bfeb-263c335f7050 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.089390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6c6-598c-49f9-bfeb-263c335f7050 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.089932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6c6-598c-49f9-bfeb-263c335f7050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.120530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ced4d6c6-598c-49f9-bfeb-263c335f7050 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.122366 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ced4d6c6-598c-49f9-bfeb-263c335f7050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.132709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 749/1540] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:49.142386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.143011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.143011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.150288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aaf49-4954-4595-bf6a-79b1d4feac7f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.151015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aaf49-4954-4595-bf6a-79b1d4feac7f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.152182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85aaf49-4954-4595-bf6a-79b1d4feac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.177674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.178936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.188958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f85aaf49-4954-4595-bf6a-79b1d4feac7f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.190649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f85aaf49-4954-4595-bf6a-79b1d4feac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.193587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 792/1541] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.204241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 750/1542] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:49.207562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.214252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.214915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.215411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.222847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa35a8-cb6c-4cc4-9857-c00f5e6f2f7c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.234772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.236386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.236386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.236386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.236669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.237052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.238737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:49.238737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:47:49.238737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:47:49.239713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.257016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.258771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.259682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 793/1543] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:49.275725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.278864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 751/1544] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:49.283626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.292144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783cc2-a029-4066-8131-3be51733b384 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.298106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d605e4a-b697-45a9-9de2-ca23834bf954 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.308781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.309435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.309718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.309992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.310273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.310647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.311372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.311909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.312410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.312820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.313533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.314149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.316038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.339538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 752/1545] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:47:49.348262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.373929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a151c48-77f0-49c5-b579-cbb204b7fe66 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.383382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.384459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.384964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.386106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.387124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.388858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:49.388858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:47:49.388858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:47:49.390535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.406774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 794/1546] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:49.412437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.477899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 753/1547] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:49.481669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.540466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6dfeb-1ada-4e6e-92a9-597205f00dd4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.626044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f80010-5025-4271-a07c-a7c7845236a6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.637349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.638355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.638749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.639035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.639746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.641717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4593f4f-d1d3-4984-827e-077b0770bb38 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.653717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.654412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.654793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.655071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.655937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.657048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:49.657048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:47:49.657048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:47:49.658324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5def50f-ba5e-4523-b420-872a5265abd1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.663498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 754/1548] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/users => generated 282 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:47:49.666749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd8d8-3e0c-4a0c-a600-c38b1b71a019 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.666749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd8d8-3e0c-4a0c-a600-c38b1b71a019 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.666749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cd8d8-3e0c-4a0c-a600-c38b1b71a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.683308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 795/1549] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/users => generated 284 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:49.683308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9495c1-e962-40a7-9f53-3d7fb2c950e3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.683308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9495c1-e962-40a7-9f53-3d7fb2c950e3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.683308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9495c1-e962-40a7-9f53-3d7fb2c950e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.686567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-125cd8d8-3e0c-4a0c-a600-c38b1b71a019 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.687809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-125cd8d8-3e0c-4a0c-a600-c38b1b71a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.695847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 755/1550] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.697104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ac6ce-3038-4f43-9502-c79ac25194e6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.697359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ac6ce-3038-4f43-9502-c79ac25194e6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.697624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ac6ce-3038-4f43-9502-c79ac25194e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.702053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ff9495c1-e962-40a7-9f53-3d7fb2c950e3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.703450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ff9495c1-e962-40a7-9f53-3d7fb2c950e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.712059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 796/1551] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.713549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0df6-a3b9-47fd-bf8e-217e03a6ee8c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.713836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0df6-a3b9-47fd-bf8e-217e03a6ee8c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.714111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e0df6-a3b9-47fd-bf8e-217e03a6ee8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.718026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-648ac6ce-3038-4f43-9502-c79ac25194e6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.719242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-648ac6ce-3038-4f43-9502-c79ac25194e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.727495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 756/1552] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.728750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.729036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.729843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.734441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-748e0df6-a3b9-47fd-bf8e-217e03a6ee8c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.735746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-748e0df6-a3b9-47fd-bf8e-217e03a6ee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.743774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 797/1553] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.745589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.745867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.746134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.750177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.751407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.759784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 757/1554] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.760727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.765143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.766404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.770610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9d9ad-68e6-4674-8bac-71044ac9a2c6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.774694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 798/1555] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.775259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.775492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.776072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.776355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.776681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.777364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.779114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bc6da-ddc6-49c2-b13f-e7b59002ae0c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.784964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20536f-d7f2-4807-a96e-1fdc8656336d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.789602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.790194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.790466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.790672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 758/1556] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:49.790778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.791410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.792431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260af1-03f3-4cb9-9f2c-d7161860e9b8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.792709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260af1-03f3-4cb9-9f2c-d7161860e9b8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.792986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d260af1-03f3-4cb9-9f2c-d7161860e9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.793565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc39ed-0c7c-4223-a9d8-265339c01538 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.803729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 799/1557] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:49.805153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d272-cd4b-4498-b942-d254144f5c03 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.805434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d272-cd4b-4498-b942-d254144f5c03 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.805706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d272-cd4b-4498-b942-d254144f5c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.813325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6d260af1-03f3-4cb9-9f2c-d7161860e9b8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.814765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d260af1-03f3-4cb9-9f2c-d7161860e9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.822762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 759/1558] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.824019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ac0a-934a-4fce-b6e5-f8cfa3b14cc2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.824300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ac0a-934a-4fce-b6e5-f8cfa3b14cc2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.824577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ac0a-934a-4fce-b6e5-f8cfa3b14cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.825634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b518d272-cd4b-4498-b942-d254144f5c03 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.826975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b518d272-cd4b-4498-b942-d254144f5c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.834882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 800/1559] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.836111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20d43-10c4-46ee-a92b-4f97a06492dc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.836390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20d43-10c4-46ee-a92b-4f97a06492dc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.836662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20d43-10c4-46ee-a92b-4f97a06492dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.843438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-24b4ac0a-934a-4fce-b6e5-f8cfa3b14cc2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.844644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-24b4ac0a-934a-4fce-b6e5-f8cfa3b14cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.852701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 760/1560] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.857206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.857206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.857206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.859415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-51b20d43-10c4-46ee-a92b-4f97a06492dc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.860532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-51b20d43-10c4-46ee-a92b-4f97a06492dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.868764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 801/1561] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.871003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.871291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.871586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.878732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.879039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:49.880137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.888516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 761/1562] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.889322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.895825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:49.897034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:49.898621 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bed01-6c4d-4889-8094-3bb3a11087dc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.903127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.903736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.904009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.904273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.905018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.908927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 802/1563] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:49.908927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.909140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.915192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025d6d-47de-44f9-9f27-e59b2cdadbb3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.917501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 762/1564] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:49.918319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.920655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.921106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.921383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.921651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.922275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.923909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.927762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17de9b2-d378-4926-ac2a-7e098ff6acae admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.932370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.933445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.933715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.933978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.934341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 803/1565] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:49.934604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.934904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.935133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.936556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.944386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f211a-0d49-4dfb-9119-d0edaf70a651 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.949161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.949760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.949930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 763/1566] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:49.950096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.950433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.951180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.951307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.951409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.952955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.965590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 804/1567] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:49.966553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.967756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6b9a-d043-423d-8990-6016a63cfac9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.973019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.973424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.973684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.974013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.974670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.974974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.976481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0462-70ee-4415-8ee2-76f884fead9b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:49.976608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.981688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:49.982267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:49.982544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:49.982815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:49.983441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:49.983772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:49.985353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:49.988848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 764/1568] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:49.989768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:49.999745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efae2d0-9c64-40dc-8c13-2985e7efffec admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.007611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 805/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:50.007781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.007896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.007987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.009531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:50.019530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55758856-7186-4fa3-a2ab-b56a6cc0b4d4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.023600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 765/1570] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:50.024860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.025254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.025937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.026300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.026642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.027353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:47:50.027767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.029430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.037016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8f7fe-968d-49a1-aab3-7dc9b393c789 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.043226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.044067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.044406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.044727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 806/1571] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:47:50.044862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] PATH_INFO: `/v3/projects/db320b58d4ca4141969bbadc1c130977/users/360996cca7994cbd91d473bef826f497/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.045677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.056510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2ca58-6cf6-4542-bd9c-02772079f7f3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.061701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db320b58d4ca4141969bbadc1c130977, user_id=360996cca7994cbd91d473bef826f497, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.061838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.062522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.062803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.062934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:50.062934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:47:50.062934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:47:50.063207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] PATH_INFO: `/v3/projects/df5c174b84d04955a4c1f795e37592db/users/46b1d010706948f29ae2affafc989958/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.065158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.078354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5c174b84d04955a4c1f795e37592db, user_id=46b1d010706948f29ae2affafc989958, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.078901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 766/1572] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:50 2022] PUT /identity/v3/projects/db320b58d4ca4141969bbadc1c130977/users/360996cca7994cbd91d473bef826f497/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:50.079789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.081298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.097723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 807/1573] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:50 2022] PUT /identity/v3/projects/df5c174b84d04955a4c1f795e37592db/users/46b1d010706948f29ae2affafc989958/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:50.098658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.157885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccc534-9228-455f-ae64-99481233457c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.162176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a5df-f53c-4a52-ab81-f8ef3460ea65 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.162626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.163194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.163389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.163697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.164517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.166432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.167470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.168260 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.168537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.168803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.169413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.171361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.195168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 767/1574] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.196057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.200254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 808/1575] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.201339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.263646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8028b-fa1d-463e-b03d-a7c471929d7c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.265729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a248-39d9-4ce9-b7c3-34a328c9f9db admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.268480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.269078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.269362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.269627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.270241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.270846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.271455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.271767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.272036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.272177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.272656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.274668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.283814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:50.300365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 809/1576] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.301163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.303940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 768/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.304656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.348129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:50.368342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858980d-00a2-4394-9e50-daea564b5b8a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.368822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860eae-4d96-4abf-8357-b025b19ae777 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.373632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.374235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.374512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.374660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.374919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.375241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.375562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.375676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.375874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.376501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.377542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416fe5e-fb6e-494c-a77c-ae5b666cfb54 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.378402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778b029-f6b2-41fd-bd0b-18b9cf533680 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.407792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 810/1578] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.407792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.407792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.407792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.408269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 769/1579] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.408798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.409222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.409499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.435582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] MFA Rules not processed for user `360996cca7994cbd91d473bef826f497`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:50.436613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] MFA Rules not processed for user `46b1d010706948f29ae2affafc989958`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:50.436990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:50.437786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:50.501130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 811/1580] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:50.503931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 770/1581] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:50.504026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.504107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.568088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e1906-adfd-456b-bbc8-6c2586586254 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.570142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfbbe-5671-41ff-8566-de2cb378f76a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.572652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.573288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.573574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.573850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.574471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.575206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.575845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.576121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.576385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.576484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.577243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.579312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.591759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:50.599180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 771/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.600020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.608223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 812/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.608880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.613279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834a794-758c-44f5-b04a-84eafc440f46 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.617874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.618189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2ec3a-af4c-44fc-bbef-84f2a85c0fe9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.618462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.618738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.619006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.619643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.622315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.623458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.624070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.624341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.624609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.625212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.627108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.643934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 772/1584] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.644856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.649459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 813/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 303 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.650262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.657369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0c3a-3cc9-437e-99dd-9dcfeb8a4db9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.661734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a61a2-b289-47ba-9c6e-143ed6ce3599 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.662126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.662718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.663008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.663283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.663930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.665939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.667286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.667931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.668207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.668474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.669088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.670991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.689780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 773/1586] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.690730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.700101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 814/1587] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.700101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.704313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61905f9-3abd-449f-90bb-cc9fe7b65a3e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.707915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1901f0-a96e-4c5d-af75-8a3af6a61790 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.709007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.709630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.709909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.710179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.710792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.712448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.712767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.713032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.713311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.713575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.714219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.716147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.718233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:50.745768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 774/1588] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.745768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.746049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 815/1589] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.746049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.806072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505e885-81a6-417d-9b62-1c51c0d30665 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.809294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0718f-4754-4889-864d-7984e8d434cd admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.811289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.811932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.812213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.812486 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.813122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.814042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.814646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.814767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.814924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.815195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.815865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.817555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.827883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 816/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.828809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.830848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 775/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.831589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.843400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dd733-f729-42b2-bfe2-c244ad7e59af admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.847222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36267469-734d-44ad-8950-b9c0d0eb93df admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.848743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.849360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.849646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.849962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.850599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.852239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.852379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.852967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.853239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.853509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.854646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.856262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.863959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 817/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.866333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.867666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 776/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.868592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.876598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb0d7-853c-4f27-8921-c83918218948 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.880190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9e7d8-0e8b-4b46-a90e-e3049f21c1c9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.881224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.881819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.882100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.882370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.883000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.884616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.884829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.885562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.885865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.886105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.886741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.888353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.896244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 818/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.897152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.902489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 777/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.902489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.907453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51527a87-4124-46af-a096-6e6ef31afd97 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.910981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158327b-8305-460c-ae5f-2011aee1ef6a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.912145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.912666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.912941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.913209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.913824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.915824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.915977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.916412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.917375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.917672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.918288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.920227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.940440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 819/1596] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.941099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.946689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 778/1597] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:50 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:50.946689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.950631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3beaaa-53bc-47ef-84c5-7442b30bc508 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.955167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb6b27-464c-4714-bef3-3e00a605311a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.955467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.956136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.956409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.956672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.957436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.959023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.959944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.960661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.961075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.961354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.962156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.964645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.970504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 820/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.971327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.976059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 779/1599] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:50.977067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:50.983504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22442708-20be-4fd2-9bf5-1b82d6b97750 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.988210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.988491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d85d91-92d5-45ce-b875-91539fc37636 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:50.988797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.989113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.989336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.989959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.991586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:50.993355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:50.994057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:50.994340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:50.994633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:50.995268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:50.996953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.002289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 821/1600] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.003188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.014397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 780/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.014397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.014697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e82da6-fa46-43c7-a9eb-14542c8a4170 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.017876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.018473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.019253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.019253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.019658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.021292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.022914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626165ea-3f96-4722-a993-6f60c4ad82b0 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.028242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.028849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.029122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.029395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.030031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.031652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.035762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 822/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.035762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.043333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69b1d9-8f69-4121-a769-3ea2a22ed7d8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.047875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 781/1603] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.047875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.051137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.051755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.052032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.052299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.052924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.054504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.057879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f27d91-ec62-4e87-a222-d515e53e81b7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.062575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.063220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.063537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.063810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.064561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.064833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 823/1604] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.066265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.068522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.078904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 782/1605] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.080395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.083010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a39ce-714b-4f30-9553-d14333e1cb73 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.091638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.092655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.093122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.093530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] PATH_INFO: `/v3/projects/ea9de30b5aa5498c9786ff1fbd24f213/users/8b302a4d67684a509ad3b57fa14c25f0/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.095689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30cb37-abb8-40fe-8f92-b63114e1cd08 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.100788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.101431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.101709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.102007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] PATH_INFO: `/v3/projects/4bca50d38140438ebd32cdcb25c65e3d/users/add01791e1374db2993c4258ec6b6d6b/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.103175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.113132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9de30b5aa5498c9786ff1fbd24f213, user_id=8b302a4d67684a509ad3b57fa14c25f0, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.115834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.120149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bca50d38140438ebd32cdcb25c65e3d, user_id=add01791e1374db2993c4258ec6b6d6b, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.124059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.128979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 824/1606] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/ea9de30b5aa5498c9786ff1fbd24f213/users/8b302a4d67684a509ad3b57fa14c25f0/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.132745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.142489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 783/1607] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/4bca50d38140438ebd32cdcb25c65e3d/users/add01791e1374db2993c4258ec6b6d6b/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.144040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.196647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.209160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd261e0-1359-4470-b168-e35f13e70a28 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.221018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352d61-baec-422a-b25d-d80bc34f88dc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.225589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.226243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.226517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.226785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] PATH_INFO: `/v3/projects/9634f36f5fde470a8bc8eff224f9bf25/users/ece940f5053747b087e04a2214e18a67/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.242147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9634f36f5fde470a8bc8eff224f9bf25, user_id=ece940f5053747b087e04a2214e18a67, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.245074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.258101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 784/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/9634f36f5fde470a8bc8eff224f9bf25/users/ece940f5053747b087e04a2214e18a67/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.258705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.266964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.267826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.268204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.268357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] PATH_INFO: `/v3/projects/0fb78a439e20439f8d93a4f7df3f28f8/users/7d3139d9e58f4b56a370661e2a5d4e5e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.285241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb78a439e20439f8d93a4f7df3f28f8, user_id=7d3139d9e58f4b56a370661e2a5d4e5e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.288470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.302081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 825/1609] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/0fb78a439e20439f8d93a4f7df3f28f8/users/7d3139d9e58f4b56a370661e2a5d4e5e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.303085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.325872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa56bc-cb3e-4318-baea-91a8ecb9374e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.364267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015b5c-29c1-4606-a4cd-172786a87c86 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.369160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.369845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.370123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.370397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] PATH_INFO: `/v3/projects/985647311467420a88154e4055c73589/users/a528f8b8d0654a9eb1eadc9e40b9c25e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.382596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.383251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.383552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.383827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] PATH_INFO: `/v3/projects/2fe3fa5e280d4fbd8a26cac05fe8aaa9/users/7813dcd78a38483682842ac199e1cd18/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.385130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985647311467420a88154e4055c73589, user_id=a528f8b8d0654a9eb1eadc9e40b9c25e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.389410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.398668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.398839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe3fa5e280d4fbd8a26cac05fe8aaa9, user_id=7813dcd78a38483682842ac199e1cd18, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.402102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.404725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 826/1610] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/985647311467420a88154e4055c73589/users/a528f8b8d0654a9eb1eadc9e40b9c25e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.405776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.425231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 785/1611] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/2fe3fa5e280d4fbd8a26cac05fe8aaa9/users/7813dcd78a38483682842ac199e1cd18/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.426191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.472464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269db941-bcd7-4c06-82dd-e1bf1cb7d300 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.473744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.489138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd620e9-76bf-445b-a1af-d1e6712ffd6e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.494015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.494678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.495623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.496049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] PATH_INFO: `/v3/projects/8bf44ba514ee4657a3d524d64a7ee059/users/9225bdab641d46d69d1475a96f3ec67c/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.512293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf44ba514ee4657a3d524d64a7ee059, user_id=9225bdab641d46d69d1475a96f3ec67c, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.515088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae237e-1053-4980-99a0-e81ce67e0b4b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.529441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 786/1612] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/8bf44ba514ee4657a3d524d64a7ee059/users/9225bdab641d46d69d1475a96f3ec67c/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.533535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adec9df-1ae8-4888-9673-9c24bcc3ded2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.533535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adec9df-1ae8-4888-9673-9c24bcc3ded2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.533535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adec9df-1ae8-4888-9673-9c24bcc3ded2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.534609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.535361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.535759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.536045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] PATH_INFO: `/v3/projects/e9f18a55891944a7b7e3fd9d61336c11/users/3b102f46f6834ed9817a4795f3670f03/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.551192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f18a55891944a7b7e3fd9d61336c11, user_id=3b102f46f6834ed9817a4795f3670f03, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.553891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2e95-3aa0-4529-976b-c1db7563ad76 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.560587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9adec9df-1ae8-4888-9673-9c24bcc3ded2 None None] MFA Rules not processed for user `8b302a4d67684a509ad3b57fa14c25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.561748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9adec9df-1ae8-4888-9673-9c24bcc3ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.569056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 827/1613] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/e9f18a55891944a7b7e3fd9d61336c11/users/3b102f46f6834ed9817a4795f3670f03/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.570310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8494ed-1e6a-4e15-866c-80ec7bc8a432 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.570593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8494ed-1e6a-4e15-866c-80ec7bc8a432 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.570868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8494ed-1e6a-4e15-866c-80ec7bc8a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.592964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6a8494ed-1e6a-4e15-866c-80ec7bc8a432 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.594185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6a8494ed-1e6a-4e15-866c-80ec7bc8a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.628304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 787/1614] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.629789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2260c9-d639-4d04-9a25-af3514f45cb3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.630076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2260c9-d639-4d04-9a25-af3514f45cb3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.630351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2260c9-d639-4d04-9a25-af3514f45cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.649681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5a2260c9-d639-4d04-9a25-af3514f45cb3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.651507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5a2260c9-d639-4d04-9a25-af3514f45cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.666257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 828/1615] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.668379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b36ed6-e1a6-465c-8481-79f3d1fe009a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.668725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b36ed6-e1a6-465c-8481-79f3d1fe009a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.669073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b36ed6-e1a6-465c-8481-79f3d1fe009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.688699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-67b36ed6-e1a6-465c-8481-79f3d1fe009a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.689900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-67b36ed6-e1a6-465c-8481-79f3d1fe009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.697435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 829/1616] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.698633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978c57-8174-49b1-82b9-a2a056f8a922 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.698913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978c57-8174-49b1-82b9-a2a056f8a922 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.699178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978c57-8174-49b1-82b9-a2a056f8a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.720303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 788/1617] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.721757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.722053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.722326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.731276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-aa978c57-8174-49b1-82b9-a2a056f8a922 None None] MFA Rules not processed for user `a528f8b8d0654a9eb1eadc9e40b9c25e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.732504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-aa978c57-8174-49b1-82b9-a2a056f8a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.741921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:51.743105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.743966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.751672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 789/1618] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.752696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.762892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05e169-faf2-458d-a9aa-e7b996d2c106 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.767658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.768260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.768537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.768808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.769471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.771077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.781841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 790/1619] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.782632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.792624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0abe6-3deb-4895-9561-33ec4fb9e770 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.797381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.797992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.798296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.798538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.799160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.800795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.803961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 830/1620] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:51.805286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013705-5080-403f-97ea-54f6048b833f None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.805570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013705-5080-403f-97ea-54f6048b833f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.805852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82013705-5080-403f-97ea-54f6048b833f None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.807203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 831/1621] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 17:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:51.808051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82013705-5080-403f-97ea-54f6048b833f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.809715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-82013705-5080-403f-97ea-54f6048b833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.812078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 791/1622] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:51.812997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.814800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:51.849990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:51.874629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82013705-5080-403f-97ea-54f6048b833f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.876309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fb5da-34d0-4730-9a0f-f521ee6e23d0 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.878514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.879203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.879595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.879872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.880032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.880631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.880919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.881178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.889312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.890056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.890740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.891488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.900144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 832/1623] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:51.901474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.901628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 792/1624] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:51.902348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.909268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ae1f5-d4ba-49e1-bf37-87eb37bee84e service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.910327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f57bff-d3fe-40a8-8467-853d227cc0ec service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.913097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.913694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.913971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.914271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.915123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.915779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.916062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.916334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.924400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.925863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.927301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.928786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.935912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 833/1625] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:51.936540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.938883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 793/1626] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:51.939677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:51.945964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b38e01-54aa-46cd-a3b5-c7094a0c8118 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.949723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4808-ecf2-4b44-999b-b1b2d92dfc32 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:51.951592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.952502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.953157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.953694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] PATH_INFO: `/v3/projects/8bf44ba514ee4657a3d524d64a7ee059/users/9225bdab641d46d69d1475a96f3ec67c/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.955310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:51.955959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.956238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.956510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:51.956734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf44ba514ee4657a3d524d64a7ee059, user_id=9225bdab641d46d69d1475a96f3ec67c, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.957370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:51.959352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.960022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508afec9-3bcf-4dfb-b685-64b38fcf1efd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:51.975940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 834/1627] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:51 2022] PUT /identity/v3/projects/8bf44ba514ee4657a3d524d64a7ee059/users/9225bdab641d46d69d1475a96f3ec67c/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:51.977529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:51.977824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:51.978101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.009860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 794/1628] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/projects => generated 405 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:52.011346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.035713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:52.037479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:52.100510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0db6c4-5adc-4bac-b0f7-14b06d831054 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.107849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.108534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.108812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.109086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] PATH_INFO: `/v3/projects/e9f18a55891944a7b7e3fd9d61336c11/users/3b102f46f6834ed9817a4795f3670f03/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.111041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f18a55891944a7b7e3fd9d61336c11, user_id=3b102f46f6834ed9817a4795f3670f03, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.114685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882757a5-40e6-474c-8d2d-becc22b7133c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.128702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 795/1629] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:52 2022] PUT /identity/v3/projects/e9f18a55891944a7b7e3fd9d61336c11/users/3b102f46f6834ed9817a4795f3670f03/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:52.129866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f944-c85a-42f6-8e8a-732b8d793f2a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.130122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f944-c85a-42f6-8e8a-732b8d793f2a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.130389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f944-c85a-42f6-8e8a-732b8d793f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.156757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 835/1630] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 17:47:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:52.159100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.175655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a91f2-2d31-412e-985d-c8c0d26cebc3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.180606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1a45f944-c85a-42f6-8e8a-732b8d793f2a None None] MFA Rules not processed for user `9225bdab641d46d69d1475a96f3ec67c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:52.181868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a45f944-c85a-42f6-8e8a-732b8d793f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:52.184717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:52.271793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 796/1631] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:52.272923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.273209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.273484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.278456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.280576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.281819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.283619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.284514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.301067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.308923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] MFA Rules not processed for user `3b102f46f6834ed9817a4795f3670f03`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:52.310168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:52.321304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:52.349375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 836/1632] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:52 2022] POST /identity/v3/users => generated 309 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:52.351645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.402062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 797/1633] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:52.403016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.469422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fb5de-16e0-4ff2-ba4c-cba4c50bcda6 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.478883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.479702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.480190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.480560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.529031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3ff33-9e42-430a-98db-eab565f66db7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.535228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.536350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.550213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.550654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.569616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.571287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.576909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.578970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.591187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 798/1634] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:52.592358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.603461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 837/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:52.604802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.606260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01d241-4f57-40f0-b093-90cdcf1d8b21 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.616301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d02c71-7233-4f0d-b92d-279ed2974fb8 service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.621387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.622127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.622702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.623215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.623930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.624945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.626046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.628090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.628820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.632879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.638196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.640733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.656921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 838/1636] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:52.669362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 799/1637] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:52.670744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.686911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bef5ef-ee2f-405d-867e-ffa7161a7833 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.700749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.702769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.703411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.704357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] PATH_INFO: `/v3/projects/ec21fe867bb343ddaa2873b89f666ce7/users/218d28fe81714bc6bf34fd1c186f3c46/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.747323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec21fe867bb343ddaa2873b89f666ce7, user_id=218d28fe81714bc6bf34fd1c186f3c46, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.750008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.750529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.763336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:52.770698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057cbb-d1ab-4907-a8c1-a490293058fc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.775605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 839/1638] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:52 2022] PUT /identity/v3/projects/ec21fe867bb343ddaa2873b89f666ce7/users/218d28fe81714bc6bf34fd1c186f3c46/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:47:52.776844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:52.851973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d5bd6-3fd1-4f95-b6c5-f96625a21c43 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:52.857441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.858071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.858347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.858963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.916128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:52.918380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:52.918660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:52.918921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:52.962889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.965128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:52.985528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 840/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:52.985928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:52.997710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:52.999047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:53.000267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ef12d-7916-4aab-87ff-9b28982c0318 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:53.015358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 800/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:53.017746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227b42-9bd7-449f-80dd-05bccf30dbe6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.018041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227b42-9bd7-449f-80dd-05bccf30dbe6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.018316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227b42-9bd7-449f-80dd-05bccf30dbe6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.019824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 801/1641] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 17:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:53.025953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.026782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.027107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.049530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:53.051103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:53.143741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 802/1642] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:47:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:47:53.217696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a717f-0a54-4107-89ef-01b54005636b service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:53.224659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:53.225288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.225568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.225925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.226734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:53.230814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe5b46-ff6e-4fda-8c2d-af9ecb944ffd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:53.277827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 841/1643] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:53.279071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:53.298034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c334-d861-458b-9c97-b9b4d1d39a4e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:53.316934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:53.317686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.317970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.318242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] PATH_INFO: `/v3/projects/ec21fe867bb343ddaa2873b89f666ce7/users/218d28fe81714bc6bf34fd1c186f3c46/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.322520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec21fe867bb343ddaa2873b89f666ce7, user_id=218d28fe81714bc6bf34fd1c186f3c46, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:53.325325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:53.366678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 803/1644] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:53 2022] PUT /identity/v3/projects/ec21fe867bb343ddaa2873b89f666ce7/users/218d28fe81714bc6bf34fd1c186f3c46/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:53.370623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0002f4-5d41-4fc3-a273-472ce4cddf74 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.371444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0002f4-5d41-4fc3-a273-472ce4cddf74 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.373292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0002f4-5d41-4fc3-a273-472ce4cddf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.451817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-fb0002f4-5d41-4fc3-a273-472ce4cddf74 None None] MFA Rules not processed for user `218d28fe81714bc6bf34fd1c186f3c46`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:53.453096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fb0002f4-5d41-4fc3-a273-472ce4cddf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:53.504271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:53.586558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79e00b-633f-41e3-b05a-5c57e048c4fc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:53.591169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:53.591958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.592671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.593357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:53.594352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.685765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 842/1645] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:53.693020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:53.694706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:53.695155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:53.780021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:53.780603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] MFA Rules not processed for user `ece940f5053747b087e04a2214e18a67`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:53.781644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:53.783826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:53.812720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 804/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:54.027630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 843/1647] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:54.032463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.053877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:54.140391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.160182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3bc55-1d93-4e35-ad04-e5adcddc9b98 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:54.173296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:54.176909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:54.179752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:54.182398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:54.210410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a96170-238a-46df-8c1d-c3392c2add1a service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:54.216516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:54.218889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:54.231097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:54.232559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:54.234562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:54.235095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:54.236241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:54.246148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:54.269834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 844/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:54 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:54.327013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 805/1649] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:47:54 2022] POST /identity/v3/projects => generated 405 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:54.331069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.351545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6aff4-eb04-4b15-a086-76a426661455 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:54.509650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:54.510382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:54.510667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:54.511026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:54.511696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:54.516872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:54.571296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 845/1650] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:47:54 2022] POST /identity/v3/users => generated 309 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:47:54.575863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.596510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6b8d9-16f1-4bb4-a058-b556789d9370 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:54.624207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:54.625111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:54.625622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:54.626143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:54.627025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:54.643734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:54.694226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 806/1651] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:54.695541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.717859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f42f9-43e8-4805-acc0-91dfac1b2973 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:54.730044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:54.730928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:54.731205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:54.731426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] PATH_INFO: `/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b/users/45a67dbcb74c4deabf595cd57c593582/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:54.769424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee67f8a6a5ba498e8856d8504cd07a6b, user_id=45a67dbcb74c4deabf595cd57c593582, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:54.772151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:54.808766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 846/1652] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:54 2022] PUT /identity/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b/users/45a67dbcb74c4deabf595cd57c593582/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:54.812434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:54.833013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9b87c-415f-48b4-9b71-08f942ceec38 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:55.021750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:55.022579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.025457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.028404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.029416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:55.031341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52c3f4-c69e-42d6-807d-771704610345 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:55.070785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 807/1653] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:47:55.075509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:55.092679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4537f64-77e2-415c-8e88-16723bba09bf admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:55.110974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:55.111699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.111986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.112257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] PATH_INFO: `/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b/users/45a67dbcb74c4deabf595cd57c593582/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.123832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee67f8a6a5ba498e8856d8504cd07a6b, user_id=45a67dbcb74c4deabf595cd57c593582, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:55.126560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85883f-a021-4833-9c1e-e8d043a0428b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:55.172228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 847/1654] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:47:55 2022] PUT /identity/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b/users/45a67dbcb74c4deabf595cd57c593582/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:55.200810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81628a1e-e7b9-4afc-8c9f-7eeec64f976b None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.201121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81628a1e-e7b9-4afc-8c9f-7eeec64f976b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.201375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81628a1e-e7b9-4afc-8c9f-7eeec64f976b None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.202797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 808/1655] 10.222.0.27 () {54 vars in 962 bytes} [Wed Aug 24 17:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:47:55.222647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3b64-51f2-4806-9e5d-132019b8f454 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.222975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3b64-51f2-4806-9e5d-132019b8f454 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.223842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b3b64-51f2-4806-9e5d-132019b8f454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.237258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:55.278665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f89b3b64-51f2-4806-9e5d-132019b8f454 None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:55.279899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f89b3b64-51f2-4806-9e5d-132019b8f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:55.374565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81628a1e-e7b9-4afc-8c9f-7eeec64f976b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:55.501069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81628a1e-e7b9-4afc-8c9f-7eeec64f976b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:55.506425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:55.507269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.507391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.507740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] PATH_INFO: `/v3/users/a528f8b8d0654a9eb1eadc9e40b9c25e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.509387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a528f8b8d0654a9eb1eadc9e40b9c25e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:55.510177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: WARNING py.warnings [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:55.510177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: warnings.warn(msg) Aug 24 17:47:55.510177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: Aug 24 17:47:55.511492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:55.523898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 848/1656] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Aug 24 17:47:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:55.533697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b0e0-567d-4a7d-94f5-4994112f2878 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.533909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b0e0-567d-4a7d-94f5-4994112f2878 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.534136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b0e0-567d-4a7d-94f5-4994112f2878 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.535661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 849/1657] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 17:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:47:55.551913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490b6f8-9cc0-4a46-87d2-70aa71bd9e82 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.552262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490b6f8-9cc0-4a46-87d2-70aa71bd9e82 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.552534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490b6f8-9cc0-4a46-87d2-70aa71bd9e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.582582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:55.620025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7490b6f8-9cc0-4a46-87d2-70aa71bd9e82 None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:55.621273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7490b6f8-9cc0-4a46-87d2-70aa71bd9e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:55.652071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a528f8b8d0654a9eb1eadc9e40b9c25e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:55.653025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a528f8b8d0654a9eb1eadc9e40b9c25e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:55.670099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 850/1658] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:47:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:55.670455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a528f8b8d0654a9eb1eadc9e40b9c25e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:55.708158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a528f8b8d0654a9eb1eadc9e40b9c25e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:55.730657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a528f8b8d0654a9eb1eadc9e40b9c25e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:55.745403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 809/1659] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:55 2022] DELETE /identity/v3/users/a528f8b8d0654a9eb1eadc9e40b9c25e => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:47:55.749891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:55.750163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:55.750437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:55.780540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:55.785684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:55.867288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:55.869057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:55.895545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 851/1660] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:55.981863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:55.983798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:56.098250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bc61e-173a-40ac-9f40-08a5ef543e08 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:56.117198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47dc4b-a65d-482a-8635-82655afc7534 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:56.117556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:56.118388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:56.120516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:56.122090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:56.124842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:56.125470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:56.125652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:56.126046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:56.154380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:56.155879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:56.180347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 852/1661] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:56.332434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:56.334534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4481b-3a6b-4a50-bb2e-7f05133100bc service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:56.355842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 810/1662] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:56.562032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:56.566701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:56.605174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b849c6-0f2c-48b6-b480-2ea1c59d1b02 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:56.617965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:56.618797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:56.620512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:56.620925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] PATH_INFO: `/v3/users/360996cca7994cbd91d473bef826f497` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:56.624395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=360996cca7994cbd91d473bef826f497)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:56.630365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:56.700712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:56.754201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360996cca7994cbd91d473bef826f497'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:56.755123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360996cca7994cbd91d473bef826f497'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:56.769024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360996cca7994cbd91d473bef826f497'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:56.808555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360996cca7994cbd91d473bef826f497'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:56.842488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360996cca7994cbd91d473bef826f497'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:56.850331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 853/1663] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:56 2022] DELETE /identity/v3/users/360996cca7994cbd91d473bef826f497 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:56.854708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:56.855613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:56.856072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:56.880140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:56.881684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:56.949673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:57.018737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 811/1664] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:57.056517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:57.058546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:57.195311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b979d-786e-46af-bf20-8560c3f4e7bf admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:57.213929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:57.214772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:57.215055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:57.217063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:57.239005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:57.240501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713f41b-1b19-47de-818c-f4d617de9358 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:57.316020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 854/1665] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:57.907955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:57.911145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:57.927514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857a935-26b5-4580-bff2-275cf5a1297f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:57.957364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:57.958035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:57.958315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:57.958727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] PATH_INFO: `/v3/users/8b302a4d67684a509ad3b57fa14c25f0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:57.960559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b302a4d67684a509ad3b57fa14c25f0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:57.962711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:58.080676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b302a4d67684a509ad3b57fa14c25f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.081420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b302a4d67684a509ad3b57fa14c25f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.100053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b302a4d67684a509ad3b57fa14c25f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.128307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b302a4d67684a509ad3b57fa14c25f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.138354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b302a4d67684a509ad3b57fa14c25f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.177778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 812/1666] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:57 2022] DELETE /identity/v3/users/8b302a4d67684a509ad3b57fa14c25f0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:47:58.179468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:58.179894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:58.180137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:58.226247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:58.229965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.388930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:58.390491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.398093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 855/1667] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:58.417717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f2a26-2ba8-433b-9840-d216d0857b6a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:58.434474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:58.435151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:58.435430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:58.436126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] PATH_INFO: `/v3/users/46b1d010706948f29ae2affafc989958` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:58.439040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b1d010706948f29ae2affafc989958)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:58.444237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:58.468047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:58.469711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.617513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b1d010706948f29ae2affafc989958'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.623699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b1d010706948f29ae2affafc989958'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.651512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b1d010706948f29ae2affafc989958'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.656824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:58.678214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956efff-861f-41d8-9ac1-ed1133b4f065 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:58.684042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b1d010706948f29ae2affafc989958'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.690898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:58.693539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:58.693814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:58.696437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:58.710065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.711962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77c9c1f2-f098-48a2-9311-52b66f9e67ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b1d010706948f29ae2affafc989958'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:58.737392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 813/1668] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:58 2022] DELETE /identity/v3/users/46b1d010706948f29ae2affafc989958 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:58.743259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a08dc7-d284-4ce4-a696-bfa786762cb3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:58.743685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a08dc7-d284-4ce4-a696-bfa786762cb3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:58.744109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a08dc7-d284-4ce4-a696-bfa786762cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:58.752206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:58.755326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:58.787433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 856/1669] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:47:58.788893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:58.793725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.817500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d7a08dc7-d284-4ce4-a696-bfa786762cb3 None None] MFA Rules not processed for user `45a67dbcb74c4deabf595cd57c593582`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:58.819036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5081664-fb50-4fcf-86c9-7412201f491c service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:58.821445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d7a08dc7-d284-4ce4-a696-bfa786762cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:58.861200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:58.970206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 814/1670] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:58.971907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:58.972371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:58.972876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:59.008937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:47:59.010321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:59.027807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:59.028442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:59.028719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:59.028990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] PATH_INFO: `/v3/projects/985647311467420a88154e4055c73589` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:59.038099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985647311467420a88154e4055c73589)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:59.039620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: WARNING py.warnings [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:47:59.039620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: warnings.warn(msg) Aug 24 17:47:59.039620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: Aug 24 17:47:59.040842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:59.109954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 815/1671] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:47:59.186824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:59.189518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:59.271863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985647311467420a88154e4055c73589', 'request_id': 'req-e155741e-9f3c-4e3f-b58a-a0599b4449d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.286963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985647311467420a88154e4055c73589', 'request_id': 'req-e155741e-9f3c-4e3f-b58a-a0599b4449d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.308011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985647311467420a88154e4055c73589', 'request_id': 'req-e155741e-9f3c-4e3f-b58a-a0599b4449d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.359622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] The token cache is being invalidate because project 985647311467420a88154e4055c73589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:47:59.359919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.362852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.368013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 857/1672] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:47:58 2022] DELETE /identity/v3/projects/985647311467420a88154e4055c73589 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:47:59.370734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:47:59.375335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:59.382309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f510c5a-d2fb-474c-b9f0-68608a9d6926 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:59.493631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:59.494234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:59.494555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:59.494827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:59.497851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:47:59.549826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155741e-9f3c-4e3f-b58a-a0599b4449d3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:47:59.559616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:59.564362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:47:59.565075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:47:59.566109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:47:59.566109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] PATH_INFO: `/v3/users/218d28fe81714bc6bf34fd1c186f3c46` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:47:59.567235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218d28fe81714bc6bf34fd1c186f3c46)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:59.569620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:59.650890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:47:59.656541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:47:59.657960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:47:59.677777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 816/1673] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:47:59.693900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218d28fe81714bc6bf34fd1c186f3c46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.694649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218d28fe81714bc6bf34fd1c186f3c46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.708123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218d28fe81714bc6bf34fd1c186f3c46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.718551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218d28fe81714bc6bf34fd1c186f3c46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.741343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218d28fe81714bc6bf34fd1c186f3c46'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:47:59.757503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 858/1674] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:47:59 2022] DELETE /identity/v3/users/218d28fe81714bc6bf34fd1c186f3c46 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:00.168153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:00.171988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:00.200030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c5e0-d109-48d3-bb7f-b03750a93623 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:00.341063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:00.341799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:00.342075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:00.342346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] PATH_INFO: `/v3/projects/db320b58d4ca4141969bbadc1c130977` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:00.344254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db320b58d4ca4141969bbadc1c130977)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:00.346626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:00.452142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db320b58d4ca4141969bbadc1c130977', 'request_id': 'req-26b642cc-c17f-4dbf-bffd-1533070f77c2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.466512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db320b58d4ca4141969bbadc1c130977', 'request_id': 'req-26b642cc-c17f-4dbf-bffd-1533070f77c2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.497532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db320b58d4ca4141969bbadc1c130977', 'request_id': 'req-26b642cc-c17f-4dbf-bffd-1533070f77c2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.513244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] The token cache is being invalidate because project db320b58d4ca4141969bbadc1c130977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:00.513510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.536727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.540889 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 817/1675] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:00 2022] DELETE /identity/v3/projects/db320b58d4ca4141969bbadc1c130977 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:00.544998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:00.548178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:00.583720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9782d91-7d60-4efa-a170-d650447c6bc0 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:00.693391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:00.694830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:00.695128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:00.695728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] PATH_INFO: `/v3/users/9225bdab641d46d69d1475a96f3ec67c` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:00.700168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9225bdab641d46d69d1475a96f3ec67c)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:00.706144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:00.711854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:00.713348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:00.743381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b642cc-c17f-4dbf-bffd-1533070f77c2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:00.770367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:00.771039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:00.771325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:00.771762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] PATH_INFO: `/v3/projects/df5c174b84d04955a4c1f795e37592db` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:00.772998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5c174b84d04955a4c1f795e37592db)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:00.774948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:00.800929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9225bdab641d46d69d1475a96f3ec67c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.801953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9225bdab641d46d69d1475a96f3ec67c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.823494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9225bdab641d46d69d1475a96f3ec67c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.881430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9225bdab641d46d69d1475a96f3ec67c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.912293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9225bdab641d46d69d1475a96f3ec67c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.949761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:00.964407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5c174b84d04955a4c1f795e37592db', 'request_id': 'req-95230781-c44c-47b3-9013-4f02bf1773fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:00.964923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 859/1676] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:00 2022] DELETE /identity/v3/users/9225bdab641d46d69d1475a96f3ec67c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:00.989369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:00.991128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:01.008482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5c174b84d04955a4c1f795e37592db', 'request_id': 'req-95230781-c44c-47b3-9013-4f02bf1773fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.040749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5c174b84d04955a4c1f795e37592db', 'request_id': 'req-95230781-c44c-47b3-9013-4f02bf1773fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.064072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:01.081120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] The token cache is being invalidate because project df5c174b84d04955a4c1f795e37592db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:01.086121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.090289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.092705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 818/1677] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:00 2022] DELETE /identity/v3/projects/df5c174b84d04955a4c1f795e37592db => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:01.109468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:01.111062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:01.209263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e516b-487e-4eaa-8d45-5fb42ca5e6d8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:01.218116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:01.218951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.219244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.219538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] PATH_INFO: `/v3/projects/ea9de30b5aa5498c9786ff1fbd24f213` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.224193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9de30b5aa5498c9786ff1fbd24f213)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:01.226435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:01.248709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95230781-c44c-47b3-9013-4f02bf1773fa admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:01.255359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:01.256465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.256946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.257242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] PATH_INFO: `/v3/users/3b102f46f6834ed9817a4795f3670f03` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.259024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b102f46f6834ed9817a4795f3670f03)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:01.261154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:01.348711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9de30b5aa5498c9786ff1fbd24f213', 'request_id': 'req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.372879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9de30b5aa5498c9786ff1fbd24f213', 'request_id': 'req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.373194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b102f46f6834ed9817a4795f3670f03'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.374259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b102f46f6834ed9817a4795f3670f03'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.389239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9de30b5aa5498c9786ff1fbd24f213', 'request_id': 'req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.394322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b102f46f6834ed9817a4795f3670f03'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.410309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] The token cache is being invalidate because project ea9de30b5aa5498c9786ff1fbd24f213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:01.410717 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.411437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b07e8726-c4e5-40e9-8b92-85bb9f9f3384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.413362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 860/1678] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:00 2022] DELETE /identity/v3/projects/ea9de30b5aa5498c9786ff1fbd24f213 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:01.424772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b102f46f6834ed9817a4795f3670f03'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.426284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.426705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.427135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.433751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c6bf5785-74e2-4aa7-ad47-9516ebba87d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b102f46f6834ed9817a4795f3670f03'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:01.442444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 819/1679] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:01 2022] DELETE /identity/v3/users/3b102f46f6834ed9817a4795f3670f03 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:01.451113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:01.452659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:01.634026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 861/1680] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:01.641677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.642024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.642235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.722749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:01.723812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:01.744246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 820/1681] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:01.745278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:01.756514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa6bd5-4c36-4040-865c-63fd981687a3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:01.772011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:01.773066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.773450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.773955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.774898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:01.776140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:01.780708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:01.812209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 862/1682] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:01.816557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:01.836144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056523f-3b68-40bc-a7da-e398ed5d8dbf None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:01.845561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:01.846743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.847131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.847566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.848433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:01.850532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:01.877054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:01.879607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:01.904145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84f60f-721e-46fe-a041-69def51a11b9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:01.924210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 821/1683] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:01 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:01.925339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:01.925956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:01.927531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:01.929641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:01.930596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] PATH_INFO: `/v3/projects/ec21fe867bb343ddaa2873b89f666ce7` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:01.936148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec21fe867bb343ddaa2873b89f666ce7)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:01.941317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.020403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:02.059939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:02.107939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec21fe867bb343ddaa2873b89f666ce7', 'request_id': 'req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.128102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec21fe867bb343ddaa2873b89f666ce7', 'request_id': 'req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.134793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628830-5996-47a1-bbc6-790ca9e72b10 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.153362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec21fe867bb343ddaa2873b89f666ce7', 'request_id': 'req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.182906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] The token cache is being invalidate because project ec21fe867bb343ddaa2873b89f666ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:02.183414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.184405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a68e06cf-a1ae-4f9e-b86d-6b254b2c6257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.190252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 863/1684] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:01 2022] DELETE /identity/v3/projects/ec21fe867bb343ddaa2873b89f666ce7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:02.193218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5bef-9a82-4492-bf98-6880f422affc None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.193667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5bef-9a82-4492-bf98-6880f422affc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.194092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a5bef-9a82-4492-bf98-6880f422affc None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.196075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 864/1685] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:48:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:02.197663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.198146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.199151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.239905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:02.245518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.268433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.269068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.269334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.269600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.270226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.277353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87332d6-39fd-4776-9bf0-d5123aa72a6a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.331625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 822/1686] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:01 2022] POST /identity/v3/users => generated 283 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:02.337160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd707-8eb0-4616-8c3b-fdbc286f4f99 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.337456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd707-8eb0-4616-8c3b-fdbc286f4f99 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.337733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd707-8eb0-4616-8c3b-fdbc286f4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.399102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-674dd707-8eb0-4616-8c3b-fdbc286f4f99 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:02.400597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-674dd707-8eb0-4616-8c3b-fdbc286f4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.414006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 865/1687] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:02.416257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:02.422395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.455854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391fdf-b474-4976-a537-04329407fd59 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.488686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.489209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.489470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.489752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.495943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.497189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.524243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 866/1688] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:02.525234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:02.528667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.554709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 823/1689] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:48:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:02.556710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.557730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.558361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.563984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ade12-f790-4caa-80ac-9a6b54436568 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.580088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.580730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.581008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.581279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] PATH_INFO: `/v3/projects/e9f18a55891944a7b7e3fd9d61336c11` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.582852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f18a55891944a7b7e3fd9d61336c11)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.584399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.590354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:02.592092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.602203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 824/1690] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:02.603364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:02.621040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae998b-eeb3-4bb2-8b51-c389b414de76 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.632195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.634775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.638304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.638772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.639508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.652333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fb7b0-8b44-48e4-ab04-a38c018a8e20 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.689797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f18a55891944a7b7e3fd9d61336c11', 'request_id': 'req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.706016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f18a55891944a7b7e3fd9d61336c11', 'request_id': 'req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.729288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f18a55891944a7b7e3fd9d61336c11', 'request_id': 'req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.737397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 825/1691] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:02.751657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c78c0-b442-4a0a-8a09-147130111aac None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.752567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c78c0-b442-4a0a-8a09-147130111aac None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.753068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c78c0-b442-4a0a-8a09-147130111aac None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.756901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 826/1692] 10.222.0.27 () {54 vars in 961 bytes} [Wed Aug 24 17:48:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:02.758018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c78c0-b442-4a0a-8a09-147130111aac None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:02.762009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] The token cache is being invalidate because project e9f18a55891944a7b7e3fd9d61336c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:02.762303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.763001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:02.768272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 867/1693] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:02 2022] DELETE /identity/v3/projects/e9f18a55891944a7b7e3fd9d61336c11 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:02.768272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:02.768272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:02.835935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:02.855740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:02.907561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c78c0-b442-4a0a-8a09-147130111aac None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.910684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974cd6-4bc0-451f-a43e-92c1e868bcb4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:02.921474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.922505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.923005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.923469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.924457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:02.924985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.926419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:02.928290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.928575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.928952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae50253-66ae-413b-869d-643d1eaedccb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:02.929421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] PATH_INFO: `/v3/projects/72e32cdebe0646a59ff4971a606a92bb/users/a4cfaecc7a7f430bb365d5621e5b8628/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.957407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 827/1694] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:02.962398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:02.966060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:02.966544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:02.980435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e32cdebe0646a59ff4971a606a92bb, user_id=a4cfaecc7a7f430bb365d5621e5b8628, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:02.987689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c7795-2187-414d-a2b6-09601cfc6ba2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:03.009005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:03.010294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.023850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 868/1695] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:02 2022] PUT /identity/v3/projects/72e32cdebe0646a59ff4971a606a92bb/users/a4cfaecc7a7f430bb365d5621e5b8628/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:03.040361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3be1-e308-4cb1-9c22-b11f77535975 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.041914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3be1-e308-4cb1-9c22-b11f77535975 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.042834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3be1-e308-4cb1-9c22-b11f77535975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.117273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-01ce3be1-e308-4cb1-9c22-b11f77535975 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:03.118596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-01ce3be1-e308-4cb1-9c22-b11f77535975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.165300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 828/1696] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Aug 24 17:48:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:03.166807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:03.170546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.320032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 869/1697] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:03.322173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.323116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.323431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.327890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95b9ac-479d-445f-9992-cb439c06114e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:03.342150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:03.342767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.343048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.343324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.343992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:03.346205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:03.404993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] MFA Rules not processed for user `a4cfaecc7a7f430bb365d5621e5b8628`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:03.407683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.409643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 829/1698] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:03 2022] POST /identity/v3/projects => generated 363 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:03.410593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:03.524660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:03.565380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb394b-aa49-4122-8678-30303d5d2970 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:03.570735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 870/1699] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:03.576554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:03.576654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:03.576991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.577106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.577324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.578654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.581223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.613775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:03.615941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ee832-0f3b-4211-9908-ac3086bd810d service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:03.638461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 830/1700] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:03.639865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.640151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.640428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.680926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:03.682869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:03.713859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:03.817082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 831/1701] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:03.821474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:03.829562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbadfe-2eb5-4089-ab75-fa3e6938ecbb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:03.838442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e9150-14d1-43c3-a788-2d9233e2a51c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:03.846691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:03.847324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.847633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.847945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.848586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:03.849792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:03.850589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:03.851392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:03.853061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:03.853614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] PATH_INFO: `/v3/projects/8bf44ba514ee4657a3d524d64a7ee059` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:03.858696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf44ba514ee4657a3d524d64a7ee059)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:03.860833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:03.904441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 832/1702] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:03 2022] POST /identity/v3/users => generated 282 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:03.905438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:03.914786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.023867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf44ba514ee4657a3d524d64a7ee059', 'request_id': 'req-8fa48d88-8d0b-4c63-b270-7aede9534aac'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.038265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf44ba514ee4657a3d524d64a7ee059', 'request_id': 'req-8fa48d88-8d0b-4c63-b270-7aede9534aac'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.041166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561c351-ff06-4532-aec7-aa8b1587dec7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.050090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf44ba514ee4657a3d524d64a7ee059', 'request_id': 'req-8fa48d88-8d0b-4c63-b270-7aede9534aac'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.068377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] The token cache is being invalidate because project 8bf44ba514ee4657a3d524d64a7ee059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:04.068865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.069767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.074929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 871/1703] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:03 2022] DELETE /identity/v3/projects/8bf44ba514ee4657a3d524d64a7ee059 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:04.076141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.079558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.149400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.150319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.150319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.151049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.156446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.195127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa48d88-8d0b-4c63-b270-7aede9534aac admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.206750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.207516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.209010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.209372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.211988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:04.212305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.216051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.218027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:04.252475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.254629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928171df-ede8-4bc8-9ef6-209e77e620b1 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.260081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 872/1704] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:04.270373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.271315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 833/1705] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:04.272522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.272899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.273183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.276180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.301197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:04.303860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.314178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 834/1706] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:04.315427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.321411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407902f-91fa-49a2-80f9-fef6d3bb55d7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.322389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.350253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.351071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.351790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.352234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.354136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcda48-550d-41cf-a222-d0e741998aba None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.355647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.361779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.369964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.370632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.370912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.371328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] PATH_INFO: `/v3/users/ece940f5053747b087e04a2214e18a67` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.372722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece940f5053747b087e04a2214e18a67)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.374738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.385219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 873/1707] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:04.390636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.425727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4206d-c3e6-409a-8e2c-6ef537db3089 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.435011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.450823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.451317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.451439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.460050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.462815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740f7f2-1e19-4641-9ca1-e48878d505d1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.508171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece940f5053747b087e04a2214e18a67'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.516060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece940f5053747b087e04a2214e18a67'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.543452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 874/1708] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:04 2022] POST /identity/v3/projects => generated 399 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:04.545295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.545583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.545875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.546957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece940f5053747b087e04a2214e18a67'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.577340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece940f5053747b087e04a2214e18a67'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.593325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-00e67146-b7f8-4ada-9ea1-e788da2c19f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece940f5053747b087e04a2214e18a67'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:04.599285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:04.605222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 835/1709] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:04 2022] DELETE /identity/v3/users/ece940f5053747b087e04a2214e18a67 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:04.612365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.616106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.616399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.616876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.686702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:04.711237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] MFA Rules not processed for user `add01791e1374db2993c4258ec6b6d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:04.715644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.736313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 875/1710] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:04.737866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.747307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.773371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be85800-5fb5-4c48-935d-30e9890a9756 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.790625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.791609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.792672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.793247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.795472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.809670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.841989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 876/1711] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:04.842783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.847012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:04.864839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87a4f2-05a6-4b57-b529-48dc5d8f94cc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.870047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.870988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.870988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.871262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.872245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.874468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.924401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 877/1712] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:04 2022] POST /identity/v3/users => generated 300 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:04.925516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.938405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76541a-c4df-4f20-b722-09ccf2f2b55a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.943346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 836/1713] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:04.945248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.946182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.947893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.948439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.948719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.949373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:04.949678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:04.952969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:04.971805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee26f8-a9f4-4028-8e9f-5f797dfe7663 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.974932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 878/1714] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:04.975949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:04.988908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148a6cf-1406-427f-9966-740438fd7b3a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:04.991769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.993597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.994173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.995114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] PATH_INFO: `/v3/projects/e27756b7d36a4d2ea88b9a4760dd640b/users/90c0e204ba64402da63a3ecd5987d279/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.996103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:04.997125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:04.998223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:04.998223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:04.998576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.001130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c435c-cbe6-4786-826c-d2f2a5132940 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.015853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 879/1715] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:05.020013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.021626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27756b7d36a4d2ea88b9a4760dd640b, user_id=90c0e204ba64402da63a3ecd5987d279, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.022493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.023889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.033535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.071527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 837/1716] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:04 2022] PUT /identity/v3/projects/e27756b7d36a4d2ea88b9a4760dd640b/users/90c0e204ba64402da63a3ecd5987d279/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:05.071527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.081758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] MFA Rules not processed for user `add01791e1374db2993c4258ec6b6d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:05.084620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.117562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:05.221473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 880/1717] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:05.222422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.289994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e8fe-ec1e-438b-bec1-949950209397 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:05.301768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:05.302585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.303992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.304519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.306501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.309795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.332126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c8be-d9ef-40f2-8830-13f39dd2c377 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:05.338565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:05.340209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.340656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.341368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.342364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.344471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027d99c-2571-4ad1-8e4a-cbc667b72438 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.359297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 881/1718] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:05.361136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.361747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.362151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.377827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 838/1719] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:05 2022] POST /identity/v3/projects => generated 367 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:05.378959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.383318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.409897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] MFA Rules not processed for user `90c0e204ba64402da63a3ecd5987d279`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:05.411577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.429315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:05.491719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8777dd-1d61-416a-be03-405c9aecb6f6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:05.524314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 882/1720] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:05.525406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.528643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.541975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:05.542639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.542919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.543192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] PATH_INFO: `/v3/projects/9634f36f5fde470a8bc8eff224f9bf25` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.547068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9634f36f5fde470a8bc8eff224f9bf25)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.549152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.651311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9634f36f5fde470a8bc8eff224f9bf25', 'request_id': 'req-ecad6501-b7df-44db-a99f-7f68c46c1227'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:05.655781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1986eb-fc60-4c14-ba9a-5bcec97d3a5e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:05.662128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9634f36f5fde470a8bc8eff224f9bf25', 'request_id': 'req-ecad6501-b7df-44db-a99f-7f68c46c1227'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:05.684831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9634f36f5fde470a8bc8eff224f9bf25', 'request_id': 'req-ecad6501-b7df-44db-a99f-7f68c46c1227'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:05.703145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] The token cache is being invalidate because project 9634f36f5fde470a8bc8eff224f9bf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:05.703653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:05.705716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:05.709000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 839/1721] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:05 2022] DELETE /identity/v3/projects/9634f36f5fde470a8bc8eff224f9bf25 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:05.709870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.717483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.784564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:05.836144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:05.836991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.839388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.839755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.843991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:05.886032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad6501-b7df-44db-a99f-7f68c46c1227 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:05.899385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:05.900143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:05.900963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:05.902365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] PATH_INFO: `/v3/projects/948dfbc82d8046048d66838e1b703a6e/users/927902301e4743f3abf7a1486e6e364b/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:05.949860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948dfbc82d8046048d66838e1b703a6e, user_id=927902301e4743f3abf7a1486e6e364b, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.953619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.966729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:05.981148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 840/1722] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:05 2022] PUT /identity/v3/projects/948dfbc82d8046048d66838e1b703a6e/users/927902301e4743f3abf7a1486e6e364b/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:05.981521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:05.982219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:05.983513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:05.994504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.022433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 883/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:06.024781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:06.028149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.092860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8d6d-e662-405b-9929-28dd42cb029e service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:06.185240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0999-0e38-43c1-8c49-2338c5940ef2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:06.192210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:06.192972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.193353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.193995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.194497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:06.199227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:06.244080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 841/1724] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:05 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:06.246837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:06.250127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.269851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686de4d7-7d67-4d7c-96dc-bd9271896a00 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:06.273476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:06.274114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.274396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.274666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.275364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:06.277235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:06.278334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd794fe-8571-44d5-8db9-9eaeca2c9e32 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:06.278465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.280191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.280618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] PATH_INFO: `/v3/users/45a67dbcb74c4deabf595cd57c593582` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.282641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a67dbcb74c4deabf595cd57c593582)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:06.285078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:06.363938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 884/1725] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:06 2022] POST /identity/v3/projects => generated 365 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:06.368453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.368701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.369694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.394629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a67dbcb74c4deabf595cd57c593582'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:06.396499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a67dbcb74c4deabf595cd57c593582'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:06.415691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a67dbcb74c4deabf595cd57c593582'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:06.443217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] MFA Rules not processed for user `927902301e4743f3abf7a1486e6e364b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:06.446328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.451177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a67dbcb74c4deabf595cd57c593582'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:06.467633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a67dbcb74c4deabf595cd57c593582'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:06.486447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 842/1726] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:06 2022] DELETE /identity/v3/users/45a67dbcb74c4deabf595cd57c593582 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:06.487538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:06.495440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.565980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:06.721158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 885/1727] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:06 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:06.722276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:06.726450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.809459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f7578-d039-4355-9afb-943c5d667dbc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:06.828312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:06.833806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.837134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.838362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.839163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:06.848096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:06.912086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 843/1728] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:06.913272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:06.935287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:06.974413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ded9a-5cba-49df-b857-93a89384a256 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:06.991064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:06.993008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:06.994373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:06.996307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:06.998320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.000331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.013344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca71d-110f-4f3f-bbfe-8f27a4d5d02b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.024392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.025027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.025361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.026131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.026774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.028785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.050668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:07.076582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 844/1729] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:06 2022] POST /identity/v3/projects => generated 401 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:07.081862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.087468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 886/1730] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:06 2022] POST /identity/v3/users => generated 289 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:07.089459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.130483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64138a2-1d6a-48b7-a2e3-d15994ca243f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.285018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48236502-3733-4d15-b3f6-b9135f04c61f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.295762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.296379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.296656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.296924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.297575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.299184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.300464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.301086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.301370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.301797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.302952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.305129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876db0b-a475-4d89-b363-8b660dbb5b76 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.331329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 845/1731] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:07.336738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.352671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e5516-b338-4059-985a-799624e0f025 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.364185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 887/1732] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:07 2022] POST /identity/v3/users => generated 307 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:07.365756 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.365946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.366221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.378019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.378019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.378019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.378019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.378741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.380332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.410170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 846/1733] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:07.411097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.433952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55137b5a-9b8a-4b7b-84f8-16a039cdb59d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.444883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] MFA Rules not processed for user `7d3139d9e58f4b56a370661e2a5d4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:07.447561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:07.449872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.449872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.449872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.450162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] PATH_INFO: `/v3/projects/9a57da05acb94b988a01bd4078da915e/users/e77cf3fa679443928fa84bb4eb9ffd4a/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.508688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a57da05acb94b988a01bd4078da915e, user_id=e77cf3fa679443928fa84bb4eb9ffd4a, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.509918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.538795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 847/1734] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:07 2022] PUT /identity/v3/projects/9a57da05acb94b988a01bd4078da915e/users/e77cf3fa679443928fa84bb4eb9ffd4a/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:07.539674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.619155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 888/1735] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:07.620723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.639926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd254-645a-48ee-af7e-785bc45372a2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.654141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:07.678169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5960f24-1055-4848-9d65-24e67b255a66 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.700593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.703304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.703721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.705184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.707146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.712165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c3c42-aa9b-4c53-b053-410d0717609f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.763744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 848/1736] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:07.768193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.770216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.770668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.775001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.775274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 849/1737] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 17:48:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:07.776383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.776659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.776930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] PATH_INFO: `/v3/projects/03396f57370341adbefc36b632fd475d/users/26cb0d54c7a64297bbf40297913f07c0/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.778449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:07.782751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:07.810936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc2f45-fbd4-4d8d-9be9-60410e7e8ef7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:07.816577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:07.817368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.817775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.818216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] PATH_INFO: `/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.819111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03396f57370341adbefc36b632fd475d, user_id=26cb0d54c7a64297bbf40297913f07c0, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.819708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee67f8a6a5ba498e8856d8504cd07a6b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:07.821923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66109c3-66fe-4626-9f6c-f7224c3be1d5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.825235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:07.847229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:07.875356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 889/1738] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:07 2022] PUT /identity/v3/projects/03396f57370341adbefc36b632fd475d/users/26cb0d54c7a64297bbf40297913f07c0/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:07.877052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:07.877587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:07.877862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:07.975163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee67f8a6a5ba498e8856d8504cd07a6b', 'request_id': 'req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:07.983064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] MFA Rules not processed for user `e77cf3fa679443928fa84bb4eb9ffd4a`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:07.984348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:07.993371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee67f8a6a5ba498e8856d8504cd07a6b', 'request_id': 'req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:08.032744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee67f8a6a5ba498e8856d8504cd07a6b', 'request_id': 'req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:08.082348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] The token cache is being invalidate because project ee67f8a6a5ba498e8856d8504cd07a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:08.082652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:08.084564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e7d698a-09aa-4d4a-86e3-004f2063b6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:08.089516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 850/1739] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:07 2022] DELETE /identity/v3/projects/ee67f8a6a5ba498e8856d8504cd07a6b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:08.093841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a36ca-d4fa-400a-ab55-1039a7334908 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.094323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a36ca-d4fa-400a-ab55-1039a7334908 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.096047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a36ca-d4fa-400a-ab55-1039a7334908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.154973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-044a36ca-d4fa-400a-ab55-1039a7334908 None None] MFA Rules not processed for user `7d3139d9e58f4b56a370661e2a5d4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:08.163793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-044a36ca-d4fa-400a-ab55-1039a7334908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.183551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 890/1740] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:08.184686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.186080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.303538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 851/1741] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:08.304944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441224-128b-4aea-ab90-6ffc64c83d48 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.305234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441224-128b-4aea-ab90-6ffc64c83d48 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.305511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441224-128b-4aea-ab90-6ffc64c83d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.355700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2fa96-e5f3-4ae9-a0ee-6c487eaaf13b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:08.367173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:08.368304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.368648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.369509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] PATH_INFO: `/v3/projects/3306dd3df2af4307b18dae54d7f11b66/users/d7651a19ca5744f888b038df2c63ba25/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.394226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a3441224-128b-4aea-ab90-6ffc64c83d48 None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:08.395721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a3441224-128b-4aea-ab90-6ffc64c83d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.397241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3306dd3df2af4307b18dae54d7f11b66, user_id=d7651a19ca5744f888b038df2c63ba25, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:08.404641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:08.448287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 891/1742] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:08 2022] PUT /identity/v3/projects/3306dd3df2af4307b18dae54d7f11b66/users/d7651a19ca5744f888b038df2c63ba25/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:08.449585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.456275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.499285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 852/1743] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:08.506213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.507261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.507632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.526513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:08.558855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:08.562185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.637077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a97a2b-9ebe-4dea-813f-1b6de082b0e2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:08.647785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:08.648536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.648814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.649082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.649731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:08.655613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162e94-3d8d-42af-adfb-18b9bdb9213e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:08.669682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 853/1744] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:08.671182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.676554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.683231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 892/1745] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:08.684740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.685504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.685782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.697532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:08.705549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea96db8-043e-4afa-bd99-6ec4f931aaeb None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:08.715190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:08.716222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.716901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.717505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.718389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:08.721280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:08.746340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] MFA Rules not processed for user `7813dcd78a38483682842ac199e1cd18`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:08.748989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.767140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 854/1746] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:08.768931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.771149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.857405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 893/1747] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:08.857405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.871517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaddcfc-9797-4f2b-ba19-9557dc154b33 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:08.881684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdbc7e-29c2-45b2-924e-2758542aafec admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:08.889325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:08.889958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.890242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.890519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.892394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:08.964013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:08.965333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:08.966236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:08.967353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:08.968954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:08.972001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926cf2-54f9-443b-9be4-4d5927d06eea admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:08.973004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:08.975344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:08.987981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 855/1748] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:08.992684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 894/1749] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:08.998894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:08.999656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.000034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.000380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.005283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:09.029468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:09.031043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:09.044024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 895/1750] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:09.048149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.051054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:09.067463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf939-3845-45fc-8e47-228978d90c6f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.084527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.086366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.086660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.086935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] PATH_INFO: `/v3/projects/03396f57370341adbefc36b632fd475d/users/26cb0d54c7a64297bbf40297913f07c0/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.095105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03396f57370341adbefc36b632fd475d, user_id=26cb0d54c7a64297bbf40297913f07c0, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.098483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf2d78-d13f-4392-85ec-35691a43485b service nova] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.102304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.102904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.104212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.104483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.104757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.113751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.115346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.126541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 856/1751] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:09.127225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.145363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f1250-f2c6-48aa-b72f-603b413f9b00 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.159331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 896/1752] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:09 2022] PUT /identity/v3/projects/03396f57370341adbefc36b632fd475d/users/26cb0d54c7a64297bbf40297913f07c0/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:09.161288 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.266128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.267911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.269657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.270020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.270747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.273870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.307975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57a886-adc4-43e0-a130-9c3dab956c7b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.323057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 857/1753] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:09 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:09.323057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.327131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.328031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.328204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.328484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] PATH_INFO: `/v3/projects/3306dd3df2af4307b18dae54d7f11b66/users/d7651a19ca5744f888b038df2c63ba25/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.332489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3306dd3df2af4307b18dae54d7f11b66, user_id=d7651a19ca5744f888b038df2c63ba25, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.340381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9aa814-afb5-4563-8073-087e89e07645 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.343360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54d02d-b97a-4477-b859-b8b26fec5959 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.358957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.359232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.359548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.359826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.360497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:09.360808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.362458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.370646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 897/1754] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:09 2022] PUT /identity/v3/projects/3306dd3df2af4307b18dae54d7f11b66/users/d7651a19ca5744f888b038df2c63ba25/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:09.370826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.371117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.371392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.390908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:09.402452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 858/1755] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:09.402452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.432386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f2676-6029-4669-a8f4-5d0241b1fd76 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.442344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] MFA Rules not processed for user `26cb0d54c7a64297bbf40297913f07c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:09.443647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:09.575380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.576153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.581571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.582728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.582728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.584278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d455a-b060-434c-a393-c6be83c77d88 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.622911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 859/1756] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:09.623125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.623336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.623582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.646806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 898/1757] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:09.646806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.711455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] MFA Rules not processed for user `d7651a19ca5744f888b038df2c63ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:09.712576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e536ff-0b02-4f66-8625-3cbec96859c8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:09.713289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:09.726263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:09.729374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:09.729726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:09.730058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:09.733849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:09.739695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:09.759246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:09.835012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 899/1758] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:09 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:09.836055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.921590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 860/1759] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:09.922326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:09.933042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.062120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b7e64-fa6d-49e8-9eb0-44e4bbcd76de None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.062326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b4ce-96e0-4bca-b014-3cf0cb943aa8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.068000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.069097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.069097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.069339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] PATH_INFO: `/v3/projects/c0810fd4ffe24ac091122a8c2072d3af/users/c80d2dfb6428487c98d52314c7dab490/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.093890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0810fd4ffe24ac091122a8c2072d3af, user_id=c80d2dfb6428487c98d52314c7dab490, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.097014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.127383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 900/1760] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:09 2022] PUT /identity/v3/projects/c0810fd4ffe24ac091122a8c2072d3af/users/c80d2dfb6428487c98d52314c7dab490/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:10.129185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.145673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.146468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.146997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.147507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.214402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:10.233132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33dfc1-7a00-4a9c-a6f9-c5b810b4b82d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.238624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.239365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.239733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.240081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.240886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.242951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.271722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 901/1761] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:10 2022] POST /identity/v3/users => generated 282 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:10.272728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.274584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.280802 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.282951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.315384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 861/1762] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:10.316846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.322850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.350306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba7d6-aa7a-485f-bfe5-09ec43a736a3 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.419752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:10.463235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.464053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.465154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.465266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.482783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31508e21-9e73-4533-bc99-d4f1592fe6a3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.499602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.500334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.500628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.501073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.558689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.560305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0de1b-3aa1-4ee6-84c1-a1a002f664e8 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.572847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 862/1763] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:10.574368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.574707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.575049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.576789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 863/1764] 10.222.0.27 () {54 vars in 912 bytes} [Wed Aug 24 17:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:10.578015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.579923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.645309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.647050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.662871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 902/1765] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:10.675368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.693052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab464b-5908-4072-8a76-45d83ea0ef06 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.702375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.703013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.703293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.703651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.706888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.708642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554b222-93e9-4b83-9782-7bb9f781ac4c service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.723368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.724544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.724925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.725204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.725581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.726109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.728829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.734214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.758175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 903/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:10.760467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.761473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 864/1767] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:10.767389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.785708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa47f20-4862-4baf-ab3e-d44049a57c86 service placement] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.789894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd32a77-d1f5-4f33-9454-09f40e542c62 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.794583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.795221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.795514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.796052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.800035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.800176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.801085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.801556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.802013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.803233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.805128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43229a75-b7b1-4009-a4bc-c1ee0015cd9b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.829061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 904/1768] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:10.857417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.857997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.858398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.911036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:10.912604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:10.914363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:10.916739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:10.928655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 865/1769] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:10.929246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:10.934371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:10.941440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628305b9-1a2d-4769-b47e-c0575232ea68 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:10.945523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:10.946146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:10.946485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:10.946679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:10.949934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:11.052449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 905/1770] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:11.054014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:11.076890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:11.086410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128edc-27be-40d3-8af6-d7b9c5577ee4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:11.097118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:11.097883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.098162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.098488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] PATH_INFO: `/v3/projects/c0810fd4ffe24ac091122a8c2072d3af/users/c80d2dfb6428487c98d52314c7dab490/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.102175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0810fd4ffe24ac091122a8c2072d3af, user_id=c80d2dfb6428487c98d52314c7dab490, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:11.109019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:11.139700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 906/1771] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:11 2022] PUT /identity/v3/projects/c0810fd4ffe24ac091122a8c2072d3af/users/c80d2dfb6428487c98d52314c7dab490/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:11.140911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:11.159029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:11.161923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:11.202532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 866/1772] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:11.203570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:11.206857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:11.238964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0dd21-122a-40c7-aff7-b71abce0f217 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:11.305890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658df1bc-7d9f-4eb3-a795-679f304367ef admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:11.321198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:11.332721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.333048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.333325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.337727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:11.370766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270643-5d14-4f5c-af27-b3562b9fc23c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:11.394903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:11.395545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.395842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.396115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.402745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 907/1773] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:11 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:11.419428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:11.420193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.420429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.420713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.421511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:11.438590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 867/1774] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:11.440093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:11.458769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f0212-4e38-4b49-9868-56d0f15ecadb service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:11.473912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:11.475033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.475924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.476455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] PATH_INFO: `/v3/projects/26e100d1ee054e47bd03b55b32c73ea2/users/47e774ea414d4d62ab6127c3249602f0/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.533273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e100d1ee054e47bd03b55b32c73ea2, user_id=47e774ea414d4d62ab6127c3249602f0, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:11.536170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e52af-f8f3-440b-95d4-e09fd0deaa8a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:11.580562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] MFA Rules not processed for user `c80d2dfb6428487c98d52314c7dab490`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:11.582532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 868/1775] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:11 2022] PUT /identity/v3/projects/26e100d1ee054e47bd03b55b32c73ea2/users/47e774ea414d4d62ab6127c3249602f0/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:11.582914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:11.586168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:11.586885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:11.586885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:11.695389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] MFA Rules not processed for user `47e774ea414d4d62ab6127c3249602f0`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:11.702378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:11.799691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 908/1776] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:11.921863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 869/1777] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:11.926649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.049618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.081177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e9fff-cf99-4e89-b963-ebd499675030 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.085805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:12.099162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.105443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.108226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.108661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.109422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.113006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.189440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:12.200734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 909/1778] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:11 2022] POST /identity/v3/projects => generated 363 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:12.202880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.223316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4a841-b1bc-4403-891c-4c0d4e02bc1d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.359743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.360911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.362220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.362637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9d8-3d3b-468e-8ee8-1e0862ba610f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.362937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.374568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.375941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.377431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.377955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.380062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.393782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.443755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 910/1779] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:12 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:12.448666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.468847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169f1a-733f-4f1d-87c2-ffd643149e94 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.475673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.476647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.477094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.477429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.478346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.480464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.513143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 911/1780] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:12.515325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.548422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.552132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.552950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1229be1-0d0a-4cd5-aec8-b7536e6563a0 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.565650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.567406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.567406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.567406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] PATH_INFO: `/v3/projects/d06a954e518e4e89ad876bb99ed8e180/users/3c21f72d9d1a4b77906c727fa62b8a5f/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.591223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 870/1781] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:12.630078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06a954e518e4e89ad876bb99ed8e180, user_id=3c21f72d9d1a4b77906c727fa62b8a5f, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.633004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.687118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 912/1782] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:12 2022] PUT /identity/v3/projects/d06a954e518e4e89ad876bb99ed8e180/users/3c21f72d9d1a4b77906c727fa62b8a5f/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:12.692486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.722129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469baea-6adf-4920-bcf2-1d52c995498a service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.880502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.881446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.885661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.890377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.891424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.893311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da165e-8721-4a37-8f28-c36a8f475acf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:12.923126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 871/1783] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:12.924415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:12.940380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5d8a-caca-4188-bcf1-28e86b745785 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:12.964688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:12.966868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:12.968895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:12.969794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] PATH_INFO: `/v3/projects/d06a954e518e4e89ad876bb99ed8e180/users/3c21f72d9d1a4b77906c727fa62b8a5f/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:12.980585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06a954e518e4e89ad876bb99ed8e180, user_id=3c21f72d9d1a4b77906c727fa62b8a5f, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:12.985892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec479eb-5775-4c1e-95a8-1094a51be480 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:13.039856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:13.040323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:13.041629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:13.051348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 913/1784] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:12 2022] PUT /identity/v3/projects/d06a954e518e4e89ad876bb99ed8e180/users/3c21f72d9d1a4b77906c727fa62b8a5f/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:13.110118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:13.147582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] MFA Rules not processed for user `3c21f72d9d1a4b77906c727fa62b8a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:13.151189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:13.417309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 872/1785] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:13.544641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c31a-80de-460b-b6cf-e594572aa9cd None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:13.545267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c31a-80de-460b-b6cf-e594572aa9cd None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:13.545694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c31a-80de-460b-b6cf-e594572aa9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:13.561760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:13.606103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-caa5c31a-80de-460b-b6cf-e594572aa9cd None None] MFA Rules not processed for user `7d3139d9e58f4b56a370661e2a5d4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:13.613236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-caa5c31a-80de-460b-b6cf-e594572aa9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:13.723213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796aa10-ff4e-4add-aac7-4588b49db2cc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:13.732132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:13.734273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:13.734555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:13.734806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:13.768120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:13.770287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:13.783999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 914/1786] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:13.803693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 873/1787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:14.771340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1329ec-a844-48c0-aef2-c890d3fd99d6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:14.772240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1329ec-a844-48c0-aef2-c890d3fd99d6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:14.772618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1329ec-a844-48c0-aef2-c890d3fd99d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:14.942412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cb1329ec-a844-48c0-aef2-c890d3fd99d6 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:14.945095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cb1329ec-a844-48c0-aef2-c890d3fd99d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:14.975182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:15.111858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 915/1788] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:48:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:15.397438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:15.399037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:15.618525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3b6f-9467-4722-aa78-222a571bba99 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:15.619410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3b6f-9467-4722-aa78-222a571bba99 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:15.631523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3b6f-9467-4722-aa78-222a571bba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:15.699111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a71c3b6f-9467-4722-aa78-222a571bba99 None None] MFA Rules not processed for user `7813dcd78a38483682842ac199e1cd18`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:15.710311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a71c3b6f-9467-4722-aa78-222a571bba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:15.745581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a5d58-4a14-431b-9482-f5d34e9394e1 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:15.765902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:15.766861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:15.767147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:15.767420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:15.812532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:15.814664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:15.841869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 874/1789] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:48:15.958935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 916/1790] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:16.741038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:16.756425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:16.842493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51e9de-0606-4a73-a1ba-55a746136155 service swift] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:16.857137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:16.879219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:16.879541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:16.882062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] PATH_INFO: `/v3/users/47e774ea414d4d62ab6127c3249602f0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:16.886190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e774ea414d4d62ab6127c3249602f0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:16.891588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:17.044006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e774ea414d4d62ab6127c3249602f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.046279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e774ea414d4d62ab6127c3249602f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.078264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e774ea414d4d62ab6127c3249602f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.126403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e774ea414d4d62ab6127c3249602f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.143208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e774ea414d4d62ab6127c3249602f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.154101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 875/1791] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:16 2022] DELETE /identity/v3/users/47e774ea414d4d62ab6127c3249602f0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:17.161634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:17.162894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:17.163362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:17.216200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:17.218361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:17.339395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 917/1792] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:17.359565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:17.365210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:17.491704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd027b00-305f-4c0a-8966-b03e83290f8f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:17.500094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:17.500726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:17.501005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:17.501398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:17.513368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:17.514819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9606-cc88-428c-8e53-12faa7a867a9 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:17.561093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 876/1793] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:17.714805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:17.725257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:17.757183 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779c6e4-7c62-4265-bf06-95498583107d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:17.770208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:17.771156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:17.771781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:17.772266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] PATH_INFO: `/v3/users/e77cf3fa679443928fa84bb4eb9ffd4a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:17.775662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77cf3fa679443928fa84bb4eb9ffd4a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:17.780357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:17.865991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77cf3fa679443928fa84bb4eb9ffd4a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.866665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77cf3fa679443928fa84bb4eb9ffd4a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.879699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77cf3fa679443928fa84bb4eb9ffd4a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.883772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:17.902292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77cf3fa679443928fa84bb4eb9ffd4a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.934052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77cf3fa679443928fa84bb4eb9ffd4a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:17.941307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 918/1794] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:17 2022] DELETE /identity/v3/users/e77cf3fa679443928fa84bb4eb9ffd4a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:17.945988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:17.946893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:17.947940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:17.967968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:17.969174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:17.999378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:18.045734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:18.047435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:18.056611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 877/1795] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:18.065941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db278e-be05-450a-b131-4b56aafc7660 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:18.076690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:18.078210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:18.080798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:18.086231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:18.086513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:18.086783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] PATH_INFO: `/v3/projects/26e100d1ee054e47bd03b55b32c73ea2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:18.089727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e100d1ee054e47bd03b55b32c73ea2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:18.092282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:18.171314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cea57-2995-4823-8656-0d0e928a7b14 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:18.171522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e100d1ee054e47bd03b55b32c73ea2', 'request_id': 'req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.177490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e100d1ee054e47bd03b55b32c73ea2', 'request_id': 'req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.189152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e100d1ee054e47bd03b55b32c73ea2', 'request_id': 'req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.205399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] The token cache is being invalidate because project 26e100d1ee054e47bd03b55b32c73ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:18.205978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.206742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.208941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 919/1796] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:18 2022] DELETE /identity/v3/projects/26e100d1ee054e47bd03b55b32c73ea2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:18.212597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:18.214114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:18.237671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:18.238325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:18.238618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:18.238879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:18.240821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:18.293251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dd67-8f51-4524-8fd9-69d7120d6aa1 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:18.322130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:18.330176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:18.331091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:18.332961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] PATH_INFO: `/v3/users/3c21f72d9d1a4b77906c727fa62b8a5f` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:18.335591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c21f72d9d1a4b77906c727fa62b8a5f)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:18.341239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:18.367767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:18.367767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:18.379224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 878/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:18.454167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c21f72d9d1a4b77906c727fa62b8a5f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.455924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c21f72d9d1a4b77906c727fa62b8a5f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.474250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c21f72d9d1a4b77906c727fa62b8a5f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.483932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:18.494910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c21f72d9d1a4b77906c727fa62b8a5f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.535038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c21f72d9d1a4b77906c727fa62b8a5f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:18.565389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 920/1798] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:18 2022] DELETE /identity/v3/users/3c21f72d9d1a4b77906c727fa62b8a5f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:19.006064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:19.007624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.032547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b1aab-5400-4241-949c-d06a312ac276 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:19.095943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:19.132833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:19.136084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:19.142243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.147536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.148538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.148963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] PATH_INFO: `/v3/projects/9a57da05acb94b988a01bd4078da915e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.151039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a57da05acb94b988a01bd4078da915e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:19.153416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:19.198363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332786-405e-4495-ac6e-03c1532a2542 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:19.218053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:19.219959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.220542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.220984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] PATH_INFO: `/v3/projects/d06a954e518e4e89ad876bb99ed8e180` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.222749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06a954e518e4e89ad876bb99ed8e180)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:19.224928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:19.316046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06a954e518e4e89ad876bb99ed8e180', 'request_id': 'req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.318573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a57da05acb94b988a01bd4078da915e', 'request_id': 'req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.328765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06a954e518e4e89ad876bb99ed8e180', 'request_id': 'req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.336442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a57da05acb94b988a01bd4078da915e', 'request_id': 'req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.362121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06a954e518e4e89ad876bb99ed8e180', 'request_id': 'req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.371235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a57da05acb94b988a01bd4078da915e', 'request_id': 'req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.396262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] The token cache is being invalidate because project 9a57da05acb94b988a01bd4078da915e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:19.396487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] The token cache is being invalidate because project d06a954e518e4e89ad876bb99ed8e180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:19.396576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.396651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.397411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.397526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b85f679-6685-4bc1-8f2e-4a9c02ddecfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.399069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 879/1799] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:19 2022] DELETE /identity/v3/projects/9a57da05acb94b988a01bd4078da915e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:19.403627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 921/1800] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:19 2022] DELETE /identity/v3/projects/d06a954e518e4e89ad876bb99ed8e180 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:19.404881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:19.407974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.409697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf29cd-4f04-44ae-92d5-0ee968d3cf1a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.410009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf29cd-4f04-44ae-92d5-0ee968d3cf1a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.410287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf29cd-4f04-44ae-92d5-0ee968d3cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.439101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c2bf29cd-4f04-44ae-92d5-0ee968d3cf1a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:19.440498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c2bf29cd-4f04-44ae-92d5-0ee968d3cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.511947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff0e78-a16b-4ad6-94c0-4e8781835f48 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:19.567376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 922/1801] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:19.593297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.596209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.596948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.687971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:19.690303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.717638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:19.719058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.721258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.723461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] PATH_INFO: `/v3/users/c80d2dfb6428487c98d52314c7dab490` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.724452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 923/1802] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:19.725576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80d2dfb6428487c98d52314c7dab490)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:19.730294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:19.737631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:19.754404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa558c0-5811-43bb-8e2e-d55609969cf0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:19.766024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:19.768289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.769439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.770535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.771391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:19.773614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:19.778301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:19.781146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:19.800568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 924/1803] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:19.801788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:19.807291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:19.827926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159af2-4d18-4952-9c34-1a3f0f144196 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:19.836484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:19.837867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:19.838203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:19.838478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] PATH_INFO: `/v3/users/927902301e4743f3abf7a1486e6e364b` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:19.839686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=927902301e4743f3abf7a1486e6e364b)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:19.842250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:19.868317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80d2dfb6428487c98d52314c7dab490'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.869512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80d2dfb6428487c98d52314c7dab490'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.885501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80d2dfb6428487c98d52314c7dab490'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.923812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80d2dfb6428487c98d52314c7dab490'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.954947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80d2dfb6428487c98d52314c7dab490'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:19.996955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:20.002721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 880/1804] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:19 2022] DELETE /identity/v3/users/c80d2dfb6428487c98d52314c7dab490 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:20.008502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.017099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.046658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927902301e4743f3abf7a1486e6e364b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:20.048608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927902301e4743f3abf7a1486e6e364b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:20.068591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927902301e4743f3abf7a1486e6e364b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:20.089864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927902301e4743f3abf7a1486e6e364b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:20.138974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-67394e1d-3f98-4667-ae34-0b66cac8a98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927902301e4743f3abf7a1486e6e364b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:20.154792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 925/1805] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:19 2022] DELETE /identity/v3/users/927902301e4743f3abf7a1486e6e364b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:20.159293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8870ce-d514-461d-a86c-0c370acb99d8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.159805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8870ce-d514-461d-a86c-0c370acb99d8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.159898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8870ce-d514-461d-a86c-0c370acb99d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.215811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ef935-62d9-4168-8b91-45916cd5443e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.236848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3c8870ce-d514-461d-a86c-0c370acb99d8 None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:20.240556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3c8870ce-d514-461d-a86c-0c370acb99d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.373217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 926/1806] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Aug 24 17:48:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:20.377829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.378744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.378944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.379310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.380655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.381187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.381661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.382668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.392829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.444287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:20.446987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.465676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 927/1807] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:20.466814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.468640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.505186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 881/1808] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:20 2022] POST /identity/v3/projects => generated 377 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:20.506566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.508832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.511047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaa493-4040-49d4-a202-6acf41e14e5b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.614127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:20.657321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.658116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.658402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.658947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.670498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.672314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.689268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f979c-e0ec-4e43-9b3a-181d15fa8f8a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.691658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 928/1809] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:20.692600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.693969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.706925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5081a-5d85-4a07-ab21-b7bbcfc1ff0a service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.713445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.714126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.714410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.714789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.732198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.732848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.733228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.733508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.734174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.736989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.739113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.744770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.765348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 929/1810] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:20.784479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.786600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:20.801038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 882/1811] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:20 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:20.805546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.818249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda87dda-ebbb-481f-8ddc-5140b6b3e6d3 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.833087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8370e-dd98-4327-83af-6d974ff729f1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.833878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.834517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.834795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.835069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] PATH_INFO: `/v3/projects/c0810fd4ffe24ac091122a8c2072d3af` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.838762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0810fd4ffe24ac091122a8c2072d3af)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.845298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:20.845982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.847990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:20.848283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:20.848643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:20.849389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:20.855151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:20.901778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 883/1812] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:20.922710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:20.962445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081db317-219e-40fc-90c7-e7cba32243ff admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:20.982440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:20.998359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:21.000548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.001372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.002705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.009881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0810fd4ffe24ac091122a8c2072d3af', 'request_id': 'req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.013249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:21.019600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:21.026667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0810fd4ffe24ac091122a8c2072d3af', 'request_id': 'req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.057135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 884/1813] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:21.059057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:21.078565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0810fd4ffe24ac091122a8c2072d3af', 'request_id': 'req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.110500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] The token cache is being invalidate because project c0810fd4ffe24ac091122a8c2072d3af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:21.112546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.114493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09dec7ab-8c2a-4d82-8e0b-3a549b91c150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.115104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60417a11-0c78-4f08-bb05-7fd6bac02d22 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:21.121528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 930/1814] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:20 2022] DELETE /identity/v3/projects/c0810fd4ffe24ac091122a8c2072d3af => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:21.126237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396dc4-0834-4677-81bc-62f3060c29ea None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.126631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396dc4-0834-4677-81bc-62f3060c29ea None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.126911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396dc4-0834-4677-81bc-62f3060c29ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.195521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ce396dc4-0834-4677-81bc-62f3060c29ea None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:21.203837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ce396dc4-0834-4677-81bc-62f3060c29ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.297981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:21.298913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.299278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.299724 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] PATH_INFO: `/v3/projects/fd70ed66c01141f5a3c193ffe503aa8f/users/7e69529282164956a64cb4a81fa37748/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.342608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd70ed66c01141f5a3c193ffe503aa8f, user_id=7e69529282164956a64cb4a81fa37748, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:21.355247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 931/1815] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:21.358093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.358637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.359099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.362264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7654f2c-5672-44f3-81c8-9ff7a2658ab9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:21.397911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 885/1816] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:21 2022] PUT /identity/v3/projects/fd70ed66c01141f5a3c193ffe503aa8f/users/7e69529282164956a64cb4a81fa37748/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:21.409548 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:21.411856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.413152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.413649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.414765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.485264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] MFA Rules not processed for user `7e69529282164956a64cb4a81fa37748`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:21.488561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.576951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 932/1817] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:21.576951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:21.582844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.611282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4285b1f-ff16-4a74-adc9-30b59ef18fcb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:21.627596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:21.630616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.632284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.634284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] PATH_INFO: `/v3/projects/948dfbc82d8046048d66838e1b703a6e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.636696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=948dfbc82d8046048d66838e1b703a6e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:21.641726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:21.702742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 886/1818] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:21.706517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:21.742244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c0efd-1cbe-449f-ad2a-4eb76b72fa8e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:21.756207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948dfbc82d8046048d66838e1b703a6e', 'request_id': 'req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.762739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948dfbc82d8046048d66838e1b703a6e', 'request_id': 'req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.763689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:21.768258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:21.778360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948dfbc82d8046048d66838e1b703a6e', 'request_id': 'req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.809726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] The token cache is being invalidate because project 948dfbc82d8046048d66838e1b703a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:21.810828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.811825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:21.814764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 933/1819] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:21 2022] DELETE /identity/v3/projects/948dfbc82d8046048d66838e1b703a6e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:21.822016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:21.824745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.912042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:21.912758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.913159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.914900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.914900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:21.914900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:21.923690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:21.947364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 887/1820] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:21.949386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:21.953435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:21.983404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7e78-d80a-4d47-b631-93fb1c40fd0d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:21.996683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:21.997341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:21.997625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:21.997900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] PATH_INFO: `/v3/users/d7651a19ca5744f888b038df2c63ba25` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:21.999896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7651a19ca5744f888b038df2c63ba25)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.001820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.063553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf2db-196c-4363-a4d1-26c37f6a1b6a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.098453 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.102261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.103576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.105013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.107815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.120060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.152287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7651a19ca5744f888b038df2c63ba25'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.153372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7651a19ca5744f888b038df2c63ba25'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.167237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7651a19ca5744f888b038df2c63ba25'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.197122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7651a19ca5744f888b038df2c63ba25'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.213763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7651a19ca5744f888b038df2c63ba25'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.221386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 934/1821] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:21 2022] POST /identity/v3/projects => generated 375 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:22.222318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.227140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:22.237424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 888/1822] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:21 2022] DELETE /identity/v3/users/d7651a19ca5744f888b038df2c63ba25 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:22.238645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.241635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:22.402186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b5d0-b48e-481d-98c7-cef2b350bd91 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.410267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.410910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.413642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.413931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.415006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281905-5ad1-447c-985a-a11b6e875368 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.416675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.422249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.434894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.435259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.435580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.436155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.436657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.438599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.480863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 889/1823] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:22 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:22.482817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.490988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:22.503505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb41b-5d92-4153-9897-fdb90b57a6e6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.514496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 935/1824] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:22 2022] POST /identity/v3/projects => generated 385 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:22.515755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.517559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.518265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.518582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.518854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.519533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.531759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.572979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 890/1825] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:22.577669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.624815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a721-32fa-4f29-a402-87af0db64c8f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.703196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780080d0-377e-430c-b2b9-d44c0e5d1143 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.714942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.715418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.715721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.715988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.716624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.725155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.740291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:22.764955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 936/1826] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:22 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:22.765851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.769531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:22.773430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.774119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.774395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.775212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] PATH_INFO: `/v3/projects/364a77db391e40769635a65da061e545/users/677af431cbdf4d34932024641ea905b1/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.798246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d7b5c-63e4-4032-98ac-116e9a42083d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.807026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a77db391e40769635a65da061e545, user_id=677af431cbdf4d34932024641ea905b1, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.810477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.811511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.812408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.812408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.812622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] PATH_INFO: `/v3/users/a4cfaecc7a7f430bb365d5621e5b8628` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.816137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4cfaecc7a7f430bb365d5621e5b8628)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.818087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:22.844419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 891/1827] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:22 2022] PUT /identity/v3/projects/364a77db391e40769635a65da061e545/users/677af431cbdf4d34932024641ea905b1/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:22.844419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:22.932580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4cfaecc7a7f430bb365d5621e5b8628'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.934853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4cfaecc7a7f430bb365d5621e5b8628'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.952328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4cfaecc7a7f430bb365d5621e5b8628'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.967183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d000050-e0fd-4adc-a4c8-d7dc67ef5a57 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:22.972987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4cfaecc7a7f430bb365d5621e5b8628'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:22.982333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:22.982960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:22.983237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:22.983595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:22.984169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:22.985776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.012138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4cfaecc7a7f430bb365d5621e5b8628'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:23.013546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 892/1828] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:23.015711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.028690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 937/1829] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:22 2022] DELETE /identity/v3/users/a4cfaecc7a7f430bb365d5621e5b8628 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:23.028690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.029760 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:23.038060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:23.081066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc7e5e-45d6-4e86-b0b2-f7ec555fee5c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.239111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb3108-2dd8-4073-b4a4-0d530f279ff8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.260275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.260919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.261203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.261474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.262136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.263776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6477fd7-e176-4fc0-a0e2-67a0e7e220b0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.287048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 938/1830] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:23.292289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.293567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.294605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.311290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.312833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.313364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.313859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.314837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.318989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.353446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 893/1831] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:23.355903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.358805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:23.363292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:23.379943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99834-94c2-4270-83fe-e563b6e03a73 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.381181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 939/1832] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:23.382013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.393250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.394203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.394425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.394836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] PATH_INFO: `/v3/projects/edb747bf9ec049eb9a2928f7b37edc0b/users/d370befd3b844a0d833305b932702574/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.398058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:23.414084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612a06-35d4-4b4f-93af-6089580dc4bd None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.424050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.438161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb747bf9ec049eb9a2928f7b37edc0b, user_id=d370befd3b844a0d833305b932702574, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.438904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.439341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.439771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] PATH_INFO: `/v3/projects/364a77db391e40769635a65da061e545/users/677af431cbdf4d34932024641ea905b1/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.453651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a77db391e40769635a65da061e545, user_id=677af431cbdf4d34932024641ea905b1, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.454163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.457311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.502549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 940/1833] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:23 2022] PUT /identity/v3/projects/364a77db391e40769635a65da061e545/users/677af431cbdf4d34932024641ea905b1/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:23.513262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.523455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 894/1834] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:23 2022] PUT /identity/v3/projects/edb747bf9ec049eb9a2928f7b37edc0b/users/d370befd3b844a0d833305b932702574/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:23.523455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:23.524454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:23.526283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:23.725955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:23.757720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfec74-b119-4aa4-a83f-bd0c9a38d08c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.771459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.772388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.772821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.773249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.793887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23181a-978f-4a2d-988f-a1b80d7c68df admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:23.933572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.935114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b3883-80e4-4edd-b1fc-986781406bc9 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:23.963172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 895/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:23.963172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.963172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.963172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.989123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:23.989123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:23.989123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:23.989123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] PATH_INFO: `/v3/users/90c0e204ba64402da63a3ecd5987d279` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:23.990110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c0e204ba64402da63a3ecd5987d279)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:23.992216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:24.046326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] MFA Rules not processed for user `677af431cbdf4d34932024641ea905b1`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:24.048307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.111561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c0e204ba64402da63a3ecd5987d279'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.113331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c0e204ba64402da63a3ecd5987d279'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.134616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c0e204ba64402da63a3ecd5987d279'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.157467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c0e204ba64402da63a3ecd5987d279'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.209314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-261f4a61-708e-428d-a10c-36630fd4afb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c0e204ba64402da63a3ecd5987d279'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.241846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 941/1836] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:23 2022] DELETE /identity/v3/users/90c0e204ba64402da63a3ecd5987d279 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:24.241846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb567f2-884c-4b33-85b9-321522d844ad None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:24.241846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb567f2-884c-4b33-85b9-321522d844ad None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:24.242248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb567f2-884c-4b33-85b9-321522d844ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:24.255438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 896/1837] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:24.269105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:24.273080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.333753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d82db-7878-4fa9-9e61-669898c5838a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:24.365110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6fb567f2-884c-4b33-85b9-321522d844ad None None] MFA Rules not processed for user `d370befd3b844a0d833305b932702574`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:24.367966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6fb567f2-884c-4b33-85b9-321522d844ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.376905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:24.493287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:24.494610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:24.494892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:24.495163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] PATH_INFO: `/v3/projects/3306dd3df2af4307b18dae54d7f11b66` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:24.505861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3306dd3df2af4307b18dae54d7f11b66)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:24.507865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:24.510694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:24.539909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 942/1838] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:24.542031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:24.542520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:24.542803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:24.573720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:24.576752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.596620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 943/1839] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:24.597809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:24.600002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.637990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3306dd3df2af4307b18dae54d7f11b66', 'request_id': 'req-e85088c7-e634-4c93-85ba-344e435a56e2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.652818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3306dd3df2af4307b18dae54d7f11b66', 'request_id': 'req-e85088c7-e634-4c93-85ba-344e435a56e2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.671037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3306dd3df2af4307b18dae54d7f11b66', 'request_id': 'req-e85088c7-e634-4c93-85ba-344e435a56e2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.699289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] The token cache is being invalidate because project 3306dd3df2af4307b18dae54d7f11b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:24.699602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.701536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:24.711063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 897/1840] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:24 2022] DELETE /identity/v3/projects/3306dd3df2af4307b18dae54d7f11b66 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:24.714078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:24.716988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:24.810127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618e6f9-0260-41c0-83e7-e92c340b682b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:24.921528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85088c7-e634-4c93-85ba-344e435a56e2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:24.940359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:24.941004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:24.941269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:24.941551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:24.942213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:24.950000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da47c9-cde3-4241-ae4e-6744f0bd5a3b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:24.994655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:24.995509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:24.995509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:24.995835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.000274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:25.027527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 898/1841] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:24 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:25.031248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.032650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.034398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.096729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:25.097994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:25.177158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.181310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:25.181462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.224144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 944/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:25.225234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.232693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:25.311292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 899/1843] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:25.313870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.352437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46c1b4-b0eb-484e-baa3-0a59193772c5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.371942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.377925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.377925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.378174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.378740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.390544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa3aca-6f13-49a3-9214-465b5ba3c71f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.465095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b746-425b-429d-937e-c828151bd3c3 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.470494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 900/1844] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:25 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:25.471863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.474613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.474939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.515432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:25.554163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:25.561917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:25.618915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 901/1845] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:25.625401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.668418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e1b05-0114-4995-a66e-a1fbf1208e4f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.692242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.692605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.693278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.693278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.693923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.695756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.706653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.721397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.721597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.721994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.728825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:25.730862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 902/1846] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:25.736822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.774583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd97adc-5e3b-4ef3-950f-40c3fb9804c4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.788605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.792027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8d87c-6619-4dd0-8ff3-deabeed2f04c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.799298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.799682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.800032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.802736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.804690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:25.807784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.813432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.833693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 945/1847] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:25.838412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123ab3-e24d-43cd-bd76-f3c2fe410c5e None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.838788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123ab3-e24d-43cd-bd76-f3c2fe410c5e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.838973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123ab3-e24d-43cd-bd76-f3c2fe410c5e None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.840493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 946/1848] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:48:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:25.846559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123ab3-e24d-43cd-bd76-f3c2fe410c5e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.853522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 903/1849] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:25.858464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.881814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f8cb-7c90-46a5-af4d-f7f41de8d5f5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.904346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.905122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.905449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.905668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.906340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.909509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a6bfa-2e31-4194-99ee-a5a0743b0662 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.911893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123ab3-e24d-43cd-bd76-f3c2fe410c5e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:25.922307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:25.922785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.923063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.923330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.924115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:25.925814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:25.949850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 947/1850] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:25.963959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:25.980053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 904/1851] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:25 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:25.983732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d3cd-15b3-4e58-a272-d8b7e378e628 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:25.985107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d3cd-15b3-4e58-a272-d8b7e378e628 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:25.985758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4d3cd-15b3-4e58-a272-d8b7e378e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:25.987075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1beba-fad1-4a74-9649-f7c0234d3897 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.055781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6cc4d3cd-15b3-4e58-a272-d8b7e378e628 None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:26.059837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6cc4d3cd-15b3-4e58-a272-d8b7e378e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:26.212483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.213485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.214614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.215443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] PATH_INFO: `/v3/projects/4d5d57a733ea4a4b88838cbe7c7ff6b3/users/0ed3ed09ecb9476d82ff9e7e6ee71dbc/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.242787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5d57a733ea4a4b88838cbe7c7ff6b3, user_id=0ed3ed09ecb9476d82ff9e7e6ee71dbc, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.248012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.275775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 948/1852] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:25 2022] PUT /identity/v3/projects/4d5d57a733ea4a4b88838cbe7c7ff6b3/users/0ed3ed09ecb9476d82ff9e7e6ee71dbc/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:26.277559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.307432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 905/1853] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:48:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:26.308938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.309782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.310333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.374544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf995-b8d0-439d-a526-c7ccc564faba admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.383839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] MFA Rules not processed for user `0ed3ed09ecb9476d82ff9e7e6ee71dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:26.384672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.385156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:26.385788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.386294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.386782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.388195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.390396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.431448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 949/1854] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:26 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:26.437810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.451417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68feac-f20e-47a9-ad47-4f0cfd320c93 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.458761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.459624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.460482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.460869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.461777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.464951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.489574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 950/1855] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:26.490695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.505938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdc7fb-492a-4de2-b108-a2b936ce3798 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.515732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.517153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.518233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.518561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.519232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.521295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.528177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:26.535871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 951/1856] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:26.542780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.557574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5522c36-0d15-420d-a1af-1bf1f103811f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.569465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.570976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.572427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.572958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] PATH_INFO: `/v3/projects/6002a842d5d94f838f03db0cb3856c60/users/0d34f5e819574892b5592514d90f8b3a/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.596142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6002a842d5d94f838f03db0cb3856c60, user_id=0d34f5e819574892b5592514d90f8b3a, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.596758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:26.600611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34a746-9f42-4dbf-9dac-b0d36a583749 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.643879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 952/1857] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:26 2022] PUT /identity/v3/projects/6002a842d5d94f838f03db0cb3856c60/users/0d34f5e819574892b5592514d90f8b3a/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:26.648180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.648468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.651037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.671209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 906/1858] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:26.675407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.701560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cef0df-69c6-442c-a03c-6d910e6ac75e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.722609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] MFA Rules not processed for user `0d34f5e819574892b5592514d90f8b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:26.724825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:26.818307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.818949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.819227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.819530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.820317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.823752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:26.864885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 907/1859] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:26 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:26.865876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.867866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:26.897709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 953/1860] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:26.898889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:26.917872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f986cb9-1761-4d0e-ba95-e0c06650cce9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.967227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4720b7f-4fcf-4c1c-9efe-58dc61c45446 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:26.973245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:26.975139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:26.975433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:26.975930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] PATH_INFO: `/v3/projects/72e32cdebe0646a59ff4971a606a92bb` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:26.977784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e32cdebe0646a59ff4971a606a92bb)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:26.980935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.116751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e32cdebe0646a59ff4971a606a92bb', 'request_id': 'req-8e923fec-1964-464d-831f-781e8d8fb7e5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:27.137020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e32cdebe0646a59ff4971a606a92bb', 'request_id': 'req-8e923fec-1964-464d-831f-781e8d8fb7e5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:27.150371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e32cdebe0646a59ff4971a606a92bb', 'request_id': 'req-8e923fec-1964-464d-831f-781e8d8fb7e5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:27.170188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:27.177499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] The token cache is being invalidate because project 72e32cdebe0646a59ff4971a606a92bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:27.181300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:27.185123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:27.190306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 908/1861] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:26 2022] DELETE /identity/v3/projects/72e32cdebe0646a59ff4971a606a92bb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:27.197764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:27.200543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:27.200852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:27.202483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.206588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.207831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.208630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:27.216210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.291668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 954/1862] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:26 2022] POST /identity/v3/users => generated 299 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:27.291668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:27.298348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:27.407921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:27.418027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923fec-1964-464d-831f-781e8d8fb7e5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:27.431950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:27.432679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.433032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.433377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.434089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:27.445273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a40d-6f65-4712-a524-0b27d67a0cbb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.452277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d8a31-2c5c-4cb9-971b-47ec249904bf admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:27.526785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 909/1863] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:27 2022] POST /identity/v3/projects => generated 381 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:27.537908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.538216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.538504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.573614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:27.579311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:27.671955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:27.672538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.672815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.673080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.675604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:27.690446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:27.692822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.712642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 955/1864] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:27.713588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:27.715068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:27.739909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2283a-aacf-45fd-856d-9602ac818e1f service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:27.810447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 910/1865] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:27.814645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:27.854959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ddce9-b7c1-4ef4-80b2-c1ee49f96a72 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:27.861401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:27.862029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.862380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.862651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.864044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:27.866040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54cc2a-37a6-4e1d-aa7e-02486a5bfd88 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.869980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:27.870757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.883463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.883984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.884710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:27.892302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:27.934808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 956/1866] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:27.934808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:27.939695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:27.944337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:27.986603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 911/1867] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:27 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:27.990776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.025695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3d84-ab65-4c18-b49d-05e77ee75ddf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.027385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:28.033466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:28.047431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.058336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.059372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.059688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] PATH_INFO: `/v3/projects/f9fed42d80004fc6bdbe894238591c84/users/50132f84fee3458b9af4c70d4e136a51/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.078920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 957/1868] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:28.082570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.108188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fed42d80004fc6bdbe894238591c84, user_id=50132f84fee3458b9af4c70d4e136a51, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.115694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.121114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d024-4c93-4295-8026-1b659a0b4ac4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.136643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.137607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.138074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.138731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.140054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.143098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.154635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 912/1869] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:27 2022] PUT /identity/v3/projects/f9fed42d80004fc6bdbe894238591c84/users/50132f84fee3458b9af4c70d4e136a51/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:28.156640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.172574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 958/1870] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:28.175750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.199444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685707e-397c-45e5-a818-34b15165c07c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.270008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:28.285567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.286642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.287448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:28.288171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.289564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.290744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.293578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.317441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 959/1871] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:28.318322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.329997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443c761-61a8-4f52-85d0-2d72c54ffbda admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.338535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.339303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.339824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.340170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] PATH_INFO: `/v3/projects/518024a86b6d4b71805d8e072e296c08/users/127c7dc774bf4c66aafff9465574279e/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.340797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dec17e-f702-45f0-be5a-dce2c0979cb9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.354105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.356027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.357475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.359702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.360213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:28.361024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.362303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.374518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518024a86b6d4b71805d8e072e296c08, user_id=127c7dc774bf4c66aafff9465574279e, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.378425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.392537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 913/1872] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:28.399456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.405878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 960/1873] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:28 2022] PUT /identity/v3/projects/518024a86b6d4b71805d8e072e296c08/users/127c7dc774bf4c66aafff9465574279e/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:28.406715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.426963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b301ba-ccc4-4d78-b63f-c4f7e7a21c57 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.585067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea1cc3-9371-4e7c-b07c-76eca70524e8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.591098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.593653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.595314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.596511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] PATH_INFO: `/v3/projects/f9fed42d80004fc6bdbe894238591c84/users/50132f84fee3458b9af4c70d4e136a51/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.600158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.601066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fed42d80004fc6bdbe894238591c84, user_id=50132f84fee3458b9af4c70d4e136a51, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.601850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.603858 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.604311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.605126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.605632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.608499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.628762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 914/1874] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:28 2022] PUT /identity/v3/projects/f9fed42d80004fc6bdbe894238591c84/users/50132f84fee3458b9af4c70d4e136a51/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:28.628762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.630707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:28.685077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 961/1875] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:28 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:28.686255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.765075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7aff0-a295-4339-8964-f6651fb646d4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.765609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd0b13-d4f7-458a-8e34-d7399cfd2bb9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.769731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.770343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.770615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.770730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.770882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.771424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.771603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.771873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.772210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] PATH_INFO: `/v3/projects/e27756b7d36a4d2ea88b9a4760dd640b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.773286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5964fc-68bf-41d6-9f0f-a4cd02fcef71 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.773385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27756b7d36a4d2ea88b9a4760dd640b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.776172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:28.785549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 962/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:28.786794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.787182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.787418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.816076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] MFA Rules not processed for user `50132f84fee3458b9af4c70d4e136a51`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:28.817255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:28.831353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27756b7d36a4d2ea88b9a4760dd640b', 'request_id': 'req-aab0ae4c-e6d9-4d12-808a-35f267c9a326'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:28.838514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27756b7d36a4d2ea88b9a4760dd640b', 'request_id': 'req-aab0ae4c-e6d9-4d12-808a-35f267c9a326'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:28.848791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27756b7d36a4d2ea88b9a4760dd640b', 'request_id': 'req-aab0ae4c-e6d9-4d12-808a-35f267c9a326'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:28.862287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:28.862743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] The token cache is being invalidate because project e27756b7d36a4d2ea88b9a4760dd640b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:28.863248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:28.863965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:28.865941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 915/1877] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:28 2022] DELETE /identity/v3/projects/e27756b7d36a4d2ea88b9a4760dd640b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:28.866955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.869454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:28.878631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:28.922808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 963/1878] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:28.924068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:28.925753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:28.983795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ae4c-e6d9-4d12-808a-35f267c9a326 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:28.989218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:28.990163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:28.990520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:28.990985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:28.992040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:28.995098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeea5a-83f7-403d-a91d-c9fd88c265f4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.019774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be96be8-06bb-4dfb-9729-86444c5ab88e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.027106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.027551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.027868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.028140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] PATH_INFO: `/v3/projects/518024a86b6d4b71805d8e072e296c08/users/127c7dc774bf4c66aafff9465574279e/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.030589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=518024a86b6d4b71805d8e072e296c08, user_id=127c7dc774bf4c66aafff9465574279e, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.033915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.061415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 916/1879] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:28 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:29.062706 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.062985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.063330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.066044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 917/1880] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:48:29.069472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 964/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:28 2022] PUT /identity/v3/projects/518024a86b6d4b71805d8e072e296c08/users/127c7dc774bf4c66aafff9465574279e/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:29.070367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.072126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.073900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.082738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.156571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b8b9-0598-4b99-9816-62672c1fd05f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.158896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672079b-540d-42c9-ac6f-5afde041c072 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.160408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.161124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.161468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.161805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.163915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.164557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.165431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.165737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] PATH_INFO: `/v3/users/26cb0d54c7a64297bbf40297913f07c0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.167508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cb0d54c7a64297bbf40297913f07c0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.170065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.224734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cb0d54c7a64297bbf40297913f07c0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:29.225486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cb0d54c7a64297bbf40297913f07c0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:29.229649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.231107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.233326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cb0d54c7a64297bbf40297913f07c0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:29.240960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 965/1882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:29.241983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.247127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cb0d54c7a64297bbf40297913f07c0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:29.253315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d5af3-1dfd-45ff-a7ef-f6146fc0bd8c service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.254398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-ca418a2b-6db6-4806-ad26-e34889fcb025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cb0d54c7a64297bbf40297913f07c0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:29.258296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.259522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.259803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.260068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.260703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.261427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 918/1883] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:29 2022] DELETE /identity/v3/users/26cb0d54c7a64297bbf40297913f07c0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:29.262408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.262692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.262971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.268660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501c394-fc9f-4f8d-8e38-ab62d073ef1a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.287436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 966/1884] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:29.288910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.289151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.289490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.306325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:29.307745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:29.313993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:29.315233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.322008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] MFA Rules not processed for user `127c7dc774bf4c66aafff9465574279e`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:29.326338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.397713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 967/1885] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:29.398734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.401029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.435959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 919/1886] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:29.436927 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.438528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.462220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb10b-608e-4704-a447-047d49c25988 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.470563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.471337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.471437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.471721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.484066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.485910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.503995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 920/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:29.504853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.506687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.557893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1989-cd79-4035-b6cf-e2b033491199 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.566554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.567191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.567468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.569420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.570138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.571929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.585208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 968/1888] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:29.589600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.605329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad277b3-f16f-42ab-a0d0-eb6819bbcc53 service nova] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.609741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c992d2-b101-4a79-a999-5144d07b6021 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.611612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.613030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.613626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.613910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.626398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.627305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.627631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.627914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] PATH_INFO: `/v3/projects/c62a8ee5f48f499481d3522ec5005e07/users/794541c03f414d58ab6ceb94ea74b854/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.647285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62a8ee5f48f499481d3522ec5005e07, user_id=794541c03f414d58ab6ceb94ea74b854, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.650891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:29.655578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed441d-a6db-40f5-bdb3-e9cd9fbe5882 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.657172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:29.686797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 921/1889] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:29.696532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 969/1890] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:29 2022] PUT /identity/v3/projects/c62a8ee5f48f499481d3522ec5005e07/users/794541c03f414d58ab6ceb94ea74b854/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:29.708357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.708357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.708357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.747843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] MFA Rules not processed for user `794541c03f414d58ab6ceb94ea74b854`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:29.749584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.825963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.892733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9d915-bef0-4836-b3a3-2d36d8f1a932 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:29.898342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:29.900532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:29.901143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:29.901794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:29.904728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 922/1891] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:29.905521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:29.909548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:29.941891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:29.976653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:30.017421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.020556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.058686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 970/1892] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:30.059432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.061383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.083667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c5b54-5dc9-44e6-9091-2ce59f472686 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.089240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.089860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.090450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.090726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.091358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.093473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.140787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 923/1893] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:29 2022] POST /identity/v3/projects => generated 389 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:30.140787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.141198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.155407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e226066-bfae-448a-83ed-7b1e1b00150e service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.226476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbc142-d1a2-45d8-85d1-35d1b0c023fc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.232451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.233255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.233606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.233950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.238843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.239766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.240422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.240766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.240967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] PATH_INFO: `/v3/projects/03396f57370341adbefc36b632fd475d` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.242112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03396f57370341adbefc36b632fd475d)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.244722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.315254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03396f57370341adbefc36b632fd475d', 'request_id': 'req-f16e7987-e577-48e2-aab1-cc6342da5b89'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.323138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03396f57370341adbefc36b632fd475d', 'request_id': 'req-f16e7987-e577-48e2-aab1-cc6342da5b89'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.326625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.328081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.333590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03396f57370341adbefc36b632fd475d', 'request_id': 'req-f16e7987-e577-48e2-aab1-cc6342da5b89'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.339983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 924/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:30.340882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.349438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] The token cache is being invalidate because project 03396f57370341adbefc36b632fd475d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:30.349949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.350881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f16e7987-e577-48e2-aab1-cc6342da5b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.352730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 971/1895] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:30 2022] DELETE /identity/v3/projects/03396f57370341adbefc36b632fd475d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:30.358057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.358307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.358589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.359790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d4418-85d4-4277-abe9-2206770a6d6a service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.366527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:30.386120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:30.390639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.443588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:30.457217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.458489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.459037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.459581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.460521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.464160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8911c22-2065-4452-956d-5642d2765210 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.471518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 972/1896] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:30.480144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.481674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.491685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 925/1897] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/users => generated 301 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:30.491685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.491685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.491685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.496608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5457c-64c1-4a48-8205-9430caf13aa1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.501520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.502194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.502460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.502727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] PATH_INFO: `/v3/users/0d34f5e819574892b5592514d90f8b3a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.503914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d34f5e819574892b5592514d90f8b3a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.505990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.511678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:30.512917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.538689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 926/1898] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:30.538689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.540269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.553656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb0420-6ad1-4ffe-abf0-acca26dc9b71 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.558881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796820-c859-4d02-8000-304786000276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.559519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796820-c859-4d02-8000-304786000276 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.559807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796820-c859-4d02-8000-304786000276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.560080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52796820-c859-4d02-8000-304786000276 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.560859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52796820-c859-4d02-8000-304786000276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.562632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52796820-c859-4d02-8000-304786000276 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.566118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d34f5e819574892b5592514d90f8b3a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.567088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d34f5e819574892b5592514d90f8b3a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.573891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 927/1899] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:30.574760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796820-c859-4d02-8000-304786000276 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.576512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-52796820-c859-4d02-8000-304786000276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.580128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d34f5e819574892b5592514d90f8b3a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.590141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52796820-c859-4d02-8000-304786000276 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.591040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d34f5e819574892b5592514d90f8b3a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.595157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.595880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.596243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.596527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.597169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:30.597467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.599245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.608485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d34f5e819574892b5592514d90f8b3a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:30.616698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 928/1900] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:30.616698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.619150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.622807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 973/1901] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:30 2022] DELETE /identity/v3/users/0d34f5e819574892b5592514d90f8b3a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:30.622807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.633544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80ad45-61c3-43ae-ae99-1aa3ddc841f1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.702823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c524c3-7ea7-41b0-9094-2cce0272f36f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.719577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.720219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.720493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.720784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.720873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.721380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.721471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.721656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.721926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] PATH_INFO: `/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f/users/dba95cfe81db4c86bdb198e70235be15/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.723428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.739269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85790571fbe4472aa4d1206fc9e6dd0f, user_id=dba95cfe81db4c86bdb198e70235be15, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.742279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e148-639c-486e-b349-814e4b2359e8 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.752206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:30.762081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 929/1902] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:30 2022] PUT /identity/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f/users/dba95cfe81db4c86bdb198e70235be15/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:30.762242 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 974/1903] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:30.762320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677708-7312-4896-9514-29e725c26f05 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.762320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677708-7312-4896-9514-29e725c26f05 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.762320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df677708-7312-4896-9514-29e725c26f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.763531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.780291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-df677708-7312-4896-9514-29e725c26f05 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:30.781489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-df677708-7312-4896-9514-29e725c26f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.827106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670990a9-11d7-4809-92a8-64940cc677ff admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.831919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.832457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.832729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.832998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.833622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.835240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.840918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 930/1904] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:30.841919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677708-7312-4896-9514-29e725c26f05 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.848218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 975/1905] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:30.849359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.853109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df677708-7312-4896-9514-29e725c26f05 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.857681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.858277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.858556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.858827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.859061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57779-d320-4a04-988b-d1e9d9a2f33f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.859436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.861428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.863615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:30.864458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:30.864742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:30.865011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] PATH_INFO: `/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f/users/dba95cfe81db4c86bdb198e70235be15/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:30.866909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85790571fbe4472aa4d1206fc9e6dd0f, user_id=dba95cfe81db4c86bdb198e70235be15, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:30.870085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:30.886011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 931/1906] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:30 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:30.887239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.887388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 976/1907] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:30 2022] PUT /identity/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f/users/dba95cfe81db4c86bdb198e70235be15/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:30.889603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:30.889954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:30.916086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:30.968627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e06503-8172-415d-aa03-8374d95665c9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:30.996382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf62e1-1292-418f-b600-7cd9f414de94 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.007101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.008287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.008769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.009197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.010544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.013188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ec321-44a7-4f37-af77-b7138e5b226c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.061691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.061912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 977/1908] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:31.062558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.062928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.063204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.063603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.063711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.063927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.075137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.076984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.113315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] MFA Rules not processed for user `dba95cfe81db4c86bdb198e70235be15`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:31.115105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.115953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 932/1909] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:31.116663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.144413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfa944-6b4b-49f5-a84c-9b567058d931 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.158051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.158051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.158051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.158430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.158819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.160587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.191546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 933/1910] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:31.196034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.218303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21188bd2-114c-473b-9eb1-70bf3eeb1366 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.230475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.231580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.232682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.233083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] PATH_INFO: `/v3/projects/c759b16c6bf64e529122be5aa0825961/users/4b9bdaf285254851a56834d1cacd9502/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.265123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c759b16c6bf64e529122be5aa0825961, user_id=4b9bdaf285254851a56834d1cacd9502, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.272112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.277248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 978/1911] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:31.277932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.279387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.286383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:31.323770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 934/1912] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:31 2022] PUT /identity/v3/projects/c759b16c6bf64e529122be5aa0825961/users/4b9bdaf285254851a56834d1cacd9502/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:31.324678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.335303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.353157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1500b2-1543-4c5b-9c36-da56c3233d26 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.413804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.414521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.414810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.415112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.425168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.496268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564a876-4675-4c5e-8053-e1fc5dc22d06 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.503169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.504459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.504753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.505032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.548381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.549744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3de14-c9c0-48bc-8f31-5f125dc3ee3e service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.574650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 979/1913] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:31.576058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.576478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.577295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.589924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:31.596698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.600559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.611230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] MFA Rules not processed for user `4b9bdaf285254851a56834d1cacd9502`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:31.612520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.625120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 935/1914] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:31.722724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:31.729180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 980/1915] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:31.734212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.761406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:31.763670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:31.892386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414dd812-c701-41ec-a2f9-7aa690ee7547 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.904638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.905864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.906255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.906870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] PATH_INFO: `/v3/projects/6002a842d5d94f838f03db0cb3856c60` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.913319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6002a842d5d94f838f03db0cb3856c60)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.916311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:31.916587 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416208-a1ca-452b-89cb-98a9c24c5935 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:31.931929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:31.932563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:31.932873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:31.933161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:31.933738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:31.939215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.029288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 936/1916] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:32.030510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.070420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6002a842d5d94f838f03db0cb3856c60', 'request_id': 'req-658ea8e3-27bb-4e42-9650-24a7e428071b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.076710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6002a842d5d94f838f03db0cb3856c60', 'request_id': 'req-658ea8e3-27bb-4e42-9650-24a7e428071b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.095773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6002a842d5d94f838f03db0cb3856c60', 'request_id': 'req-658ea8e3-27bb-4e42-9650-24a7e428071b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.109016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ebaf6-ca1a-4747-9b38-25fd4caee2cb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.117063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] The token cache is being invalidate because project 6002a842d5d94f838f03db0cb3856c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:32.117630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.119278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.122611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 981/1917] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:31 2022] DELETE /identity/v3/projects/6002a842d5d94f838f03db0cb3856c60 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:32.123320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.124768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.202318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.203207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.203759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.204192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] PATH_INFO: `/v3/users/794541c03f414d58ab6ceb94ea74b854` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.205838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=794541c03f414d58ab6ceb94ea74b854)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.208129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.214233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ea8e3-27bb-4e42-9650-24a7e428071b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.222250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.223033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.223373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.223761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.224349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.226950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.250203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:32.253150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 982/1918] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:32 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:32.253150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.253150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.270280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58230190-a453-47b0-b554-b72ff51488ff admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.275878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794541c03f414d58ab6ceb94ea74b854'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.276741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794541c03f414d58ab6ceb94ea74b854'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.277815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.278803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.279285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.279767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] PATH_INFO: `/v3/users/127c7dc774bf4c66aafff9465574279e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.281144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=127c7dc774bf4c66aafff9465574279e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.283352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.284722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794541c03f414d58ab6ceb94ea74b854'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.298484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794541c03f414d58ab6ceb94ea74b854'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.299537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:32.306769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794541c03f414d58ab6ceb94ea74b854'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.314319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 937/1919] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:32 2022] DELETE /identity/v3/users/794541c03f414d58ab6ceb94ea74b854 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:32.315243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.316993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.339180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127c7dc774bf4c66aafff9465574279e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.339911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127c7dc774bf4c66aafff9465574279e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.352131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127c7dc774bf4c66aafff9465574279e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.363184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127c7dc774bf4c66aafff9465574279e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.382364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-63ff63b8-22bd-450e-802c-7805725afa5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127c7dc774bf4c66aafff9465574279e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:32.391669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 983/1920] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:32 2022] DELETE /identity/v3/users/127c7dc774bf4c66aafff9465574279e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:32.395501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a250a0-acf6-4945-a8e3-fa9c74b2cfa6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.395900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.396182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.396458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.418363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:32.419999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.512541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 984/1921] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:32.514042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.515317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.516257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.517071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.518112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.523151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.529271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.531334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.541056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 938/1922] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:32.542119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.550443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.563920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106938f6-4ec6-40f7-81d5-083ae6d9c4eb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.568936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.569685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.570035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.570388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] PATH_INFO: `/v3/projects/c61282fba32f4599814b39d37261de53/users/f76522a55afd4e5c914684391b2ff6ad/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.586253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61282fba32f4599814b39d37261de53, user_id=f76522a55afd4e5c914684391b2ff6ad, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.589550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.608038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 939/1923] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:32 2022] PUT /identity/v3/projects/c61282fba32f4599814b39d37261de53/users/f76522a55afd4e5c914684391b2ff6ad/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:32.611899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.616753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fb1a1-2231-4528-bad2-8bf4c4f84ac2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.643959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:32.704000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:32.713874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.715460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.716421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a0d09-b7ab-4772-b947-33513ec12df4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.717286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.717632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.724035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.724649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.724924 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.726512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.727500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.731996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.735418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.736980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec39722-55e6-4307-8939-e062cf6be4b8 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.746673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 940/1924] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:32.747753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.749180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 985/1925] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:32.759653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26c984-8a81-4d7b-82c4-ad07579b7048 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.764664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.765076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.765376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.765664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] PATH_INFO: `/v3/projects/c61282fba32f4599814b39d37261de53/users/f76522a55afd4e5c914684391b2ff6ad/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.767613 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61282fba32f4599814b39d37261de53, user_id=f76522a55afd4e5c914684391b2ff6ad, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:32.770267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:32.781527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 941/1926] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:32 2022] PUT /identity/v3/projects/c61282fba32f4599814b39d37261de53/users/f76522a55afd4e5c914684391b2ff6ad/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:32.792059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.792326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.792602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:32.821648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] MFA Rules not processed for user `f76522a55afd4e5c914684391b2ff6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:32.822877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.893799 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 986/1927] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:32.905871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.907910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.980196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:32.981660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:32.989858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f216b11-ba57-4937-ba83-f86b785cf09c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:32.994737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:32.995344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:32.995645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:32.995915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.007194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.009291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.027663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 942/1928] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:33.105700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95994fc5-1651-4e3b-84f4-eb2adf6346cb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.115654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.115654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.115654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.115654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] PATH_INFO: `/v3/projects/518024a86b6d4b71805d8e072e296c08` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.116169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518024a86b6d4b71805d8e072e296c08)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.116169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.147588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.157499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c662c79-914c-4ac1-969d-bcc4bd8b5e79 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.178464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518024a86b6d4b71805d8e072e296c08', 'request_id': 'req-6701a304-bce0-4667-8e75-62b7d8207e36'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.191095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518024a86b6d4b71805d8e072e296c08', 'request_id': 'req-6701a304-bce0-4667-8e75-62b7d8207e36'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.209893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518024a86b6d4b71805d8e072e296c08', 'request_id': 'req-6701a304-bce0-4667-8e75-62b7d8207e36'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.222370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.222996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.223275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.223618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.225406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.226627 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] The token cache is being invalidate because project 518024a86b6d4b71805d8e072e296c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:33.226921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.228089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.229887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 987/1929] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:32 2022] DELETE /identity/v3/projects/518024a86b6d4b71805d8e072e296c08 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:33.230647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.231981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.244709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:33.297659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.299393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b45eb-a926-40b2-8617-2f6f4f74a21e service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.309271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a304-bce0-4667-8e75-62b7d8207e36 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.311449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 943/1930] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:33.312665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b49d6-03bc-4a5b-9464-2a08f0f9c73b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.312950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b49d6-03bc-4a5b-9464-2a08f0f9c73b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.313331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b49d6-03bc-4a5b-9464-2a08f0f9c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.323035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:33.332778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.333661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.334011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.335788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] PATH_INFO: `/v3/projects/c62a8ee5f48f499481d3522ec5005e07` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.337086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62a8ee5f48f499481d3522ec5005e07)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.338528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9e5b49d6-03bc-4a5b-9464-2a08f0f9c73b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:33.339142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.339919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9e5b49d6-03bc-4a5b-9464-2a08f0f9c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.349878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 944/1931] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:33.358565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.358853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.359129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.385103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:33.387726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.400486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 945/1932] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:33.401437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.412595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a7e8-8c26-4f3b-90b0-52a108696a43 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.416335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62a8ee5f48f499481d3522ec5005e07', 'request_id': 'req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.426509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62a8ee5f48f499481d3522ec5005e07', 'request_id': 'req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.437874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62a8ee5f48f499481d3522ec5005e07', 'request_id': 'req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.451723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] The token cache is being invalidate because project c62a8ee5f48f499481d3522ec5005e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:33.452053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.452739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.454372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 988/1933] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:33 2022] DELETE /identity/v3/projects/c62a8ee5f48f499481d3522ec5005e07 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:33.458604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.460528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.498828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.499823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.500272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.500728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.501554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:33.502059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.504182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.519054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 946/1934] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:33.525195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.534371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.550287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61763a84-d3f3-4d23-aa73-b808665fd12e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.557136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.557973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.558064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.558391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.559079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.561213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.585092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebc0c9-90b4-41c8-a5e3-7b53dd38ee63 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.593059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.594138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.594501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.594833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] PATH_INFO: `/v3/users/dba95cfe81db4c86bdb198e70235be15` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.596003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba95cfe81db4c86bdb198e70235be15)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.600848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 947/1935] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:33.601002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.602123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.678006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:33.690133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba95cfe81db4c86bdb198e70235be15'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.691956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba95cfe81db4c86bdb198e70235be15'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.712354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da86a48-670c-4705-bd16-3011fd4f14f2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.717994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba95cfe81db4c86bdb198e70235be15'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.739298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.740335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.740882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.741216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.741671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba95cfe81db4c86bdb198e70235be15'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.742102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.744430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.775682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba95cfe81db4c86bdb198e70235be15'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:33.783967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 948/1936] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:33 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:33.784933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.786729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:33.796127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 989/1937] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:33 2022] DELETE /identity/v3/users/dba95cfe81db4c86bdb198e70235be15 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:33.804089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138ddb-a13d-47bd-aebf-cbc4fee23f8f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.864920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:33.881892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.882589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.882936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.883276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.884049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.885745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.900094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 949/1938] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:33.901831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.912765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05960-e9a4-4d2a-871c-ac83ae901c96 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.917920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.918606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.918793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.919052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.919715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.921491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c9359-8ca7-4527-b5d1-25fd9372a951 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:33.932176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 990/1939] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:33.936922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:33.953424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b62be8-8faf-4e97-b652-d9692a35e071 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:33.960256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:33.960846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:33.961134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:33.961413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] PATH_INFO: `/v3/projects/13b11d25280c42d9992c44abf1d6a6ed/users/283880308765494c941914f4c4bab305/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:33.999043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b11d25280c42d9992c44abf1d6a6ed, user_id=283880308765494c941914f4c4bab305, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:33.999043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.007552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 950/1940] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:33 2022] PUT /identity/v3/projects/13b11d25280c42d9992c44abf1d6a6ed/users/283880308765494c941914f4c4bab305/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:34.017011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.017300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.017571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.049661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] MFA Rules not processed for user `283880308765494c941914f4c4bab305`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.050896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.137596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 991/1941] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:34.149932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.151825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.212464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.212618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a568d-98a2-41f1-b522-3477bda3061d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.215715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.216820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.217433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.217710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.217975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.230987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.232402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.243349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 951/1942] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:34.265298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.288508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443274d-4ab4-40cd-b201-351e4014f318 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.293231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.293862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.294142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.294415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] PATH_INFO: `/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.295573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85790571fbe4472aa4d1206fc9e6dd0f)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.297546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.329078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e2706-f03b-4f80-b722-805acbfedb5b service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.333731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.334357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.334633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.334904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] PATH_INFO: `/v3/users/283880308765494c941914f4c4bab305` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.336064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=283880308765494c941914f4c4bab305)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.336329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:34.338646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.348973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85790571fbe4472aa4d1206fc9e6dd0f', 'request_id': 'req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.355065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85790571fbe4472aa4d1206fc9e6dd0f', 'request_id': 'req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.363754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85790571fbe4472aa4d1206fc9e6dd0f', 'request_id': 'req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.375869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] The token cache is being invalidate because project 85790571fbe4472aa4d1206fc9e6dd0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:34.376185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.376848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-09ac366b-8c9a-49ef-88ea-799d4dd3cb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.378364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 992/1943] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:34 2022] DELETE /identity/v3/projects/85790571fbe4472aa4d1206fc9e6dd0f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:34.383054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629173a4-d457-4552-a0de-24d2098284f7 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.383335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629173a4-d457-4552-a0de-24d2098284f7 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.384186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629173a4-d457-4552-a0de-24d2098284f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.388526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283880308765494c941914f4c4bab305'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.389205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283880308765494c941914f4c4bab305'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.399663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283880308765494c941914f4c4bab305'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.403809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-629173a4-d457-4552-a0de-24d2098284f7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.405013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-629173a4-d457-4552-a0de-24d2098284f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.411520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:34.412750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283880308765494c941914f4c4bab305'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.421571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f92cf8fa-fce4-4eb0-bd6e-2b8853831dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283880308765494c941914f4c4bab305'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:34.427993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 952/1944] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:34 2022] DELETE /identity/v3/users/283880308765494c941914f4c4bab305 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:34.432888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.433115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.433429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.452914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.454165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.469228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 993/1945] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:34.476928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.477249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.477536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.496772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.497994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.514380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 953/1946] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:34.524768 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.526213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.567463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 994/1947] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:34.573419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.582588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21dc9e-517d-45f4-b7a7-a18c1f977b0d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.587524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.588499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.588911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.589183 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.589822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:34.590125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.591743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.595205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b34a3-f025-484d-8751-cfc9962fdcff None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.600105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.601024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.601479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.601940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.604196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 995/1948] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:48:34.608332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.619990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.621273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083e76-5ca8-46dd-bfca-a968f13ef2f5 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.621426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.636330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.636953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.637237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.637510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.637997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 954/1949] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:34.638190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.640161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.672556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 996/1950] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/projects => generated 407 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:34.673674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.684415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1141e-71be-47a4-b050-3a4368666090 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.745890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.746389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.746664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.746934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.747990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.750218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.779222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 955/1951] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/users => generated 304 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:34.780879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.795374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1cf6-1644-44b9-9f9c-4ea16debf7e7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.799818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.800677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.800967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.801408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.801898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.804142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.819746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 997/1952] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:34.823662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.835133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309ce69-1256-4318-a376-776b8c0d9330 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.851681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.851681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.851681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.851681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.853093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.855160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c9157c-5e6b-475a-a8a3-5c298a4a0b60 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.867065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 956/1953] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:34.871835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.882335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b013ab-666f-4d18-b891-1d6997d89a75 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:34.887647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:34.888306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.888584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.888852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] PATH_INFO: `/v3/projects/1456c3fed4db4882aa65f6066ae32725/users/4e27317d68904a50b30eb5e8d7ac9f71/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.906395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1456c3fed4db4882aa65f6066ae32725, user_id=4e27317d68904a50b30eb5e8d7ac9f71, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:34.909303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:34.912588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:34.927307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 998/1954] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:34 2022] PUT /identity/v3/projects/1456c3fed4db4882aa65f6066ae32725/users/4e27317d68904a50b30eb5e8d7ac9f71/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:34.938815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:34.939168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:34.939574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:34.945806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:34.951536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:34.976263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:34.994892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] MFA Rules not processed for user `4e27317d68904a50b30eb5e8d7ac9f71`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:34.996228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.046800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005e14-b986-4d31-8350-a2c6464bd44c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.054480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.055443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.056493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.056889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] PATH_INFO: `/v3/projects/13b11d25280c42d9992c44abf1d6a6ed` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.058218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b11d25280c42d9992c44abf1d6a6ed)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.060688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.078712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 957/1955] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:34 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:35.083215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.094087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfddd58-1ce7-441d-b505-bfd7c8737093 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.099460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.100105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.100389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.100665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.102097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.103794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.125469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b11d25280c42d9992c44abf1d6a6ed', 'request_id': 'req-5041d780-98df-4a36-9fc1-79b9c3ae963f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:35.137523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b11d25280c42d9992c44abf1d6a6ed', 'request_id': 'req-5041d780-98df-4a36-9fc1-79b9c3ae963f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:35.139418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 958/1956] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:35.144124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.150451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b11d25280c42d9992c44abf1d6a6ed', 'request_id': 'req-5041d780-98df-4a36-9fc1-79b9c3ae963f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:35.169090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] The token cache is being invalidate because project 13b11d25280c42d9992c44abf1d6a6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:35.169398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:35.170128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5041d780-98df-4a36-9fc1-79b9c3ae963f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:35.175323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 999/1957] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:34 2022] DELETE /identity/v3/projects/13b11d25280c42d9992c44abf1d6a6ed => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:35.176839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2456bc-5a4e-443f-bf5e-6171fd8bdc3c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.177217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2456bc-5a4e-443f-bf5e-6171fd8bdc3c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.177493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2456bc-5a4e-443f-bf5e-6171fd8bdc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.198613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6f2456bc-5a4e-443f-bf5e-6171fd8bdc3c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:35.199878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6f2456bc-5a4e-443f-bf5e-6171fd8bdc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.231970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d19446-ae75-4c9a-9b74-b915149be85c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.236812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.237417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.237695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.237963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.238581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.240555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.264005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 959/1958] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/users => generated 311 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:35.269226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.271298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1000/1959] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:35.271521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.280613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.280828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.281096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.286601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a51de-784a-41f0-a3db-087cc565be4e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.292067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.292741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.293083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.293447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.294247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.295881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.302370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:35.303645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.309544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 960/1960] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:35.313366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1001/1961] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:35.313758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.317912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.324971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e360b0c-4bf9-429f-bb41-abfe435fb2d2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.328813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa26a2e-e0e3-4715-9de7-8de35694020c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.330851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.331539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.331821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.332089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] PATH_INFO: `/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370/users/469f3f79bd2e47bb8f5e16f29e44560c/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.334653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.335267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.335584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.335860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.336509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:35.336837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.338609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.352861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27f87b3ab7c4867aa7f3b84c5e21370, user_id=469f3f79bd2e47bb8f5e16f29e44560c, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.354496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1002/1962] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:35.355468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.356039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.365936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f476f-cc78-4ab7-a78b-8c4acb2a3e56 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.370064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 961/1963] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:35 2022] PUT /identity/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370/users/469f3f79bd2e47bb8f5e16f29e44560c/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:35.371270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:35.375388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.421090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:35.442198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.442843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.443128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.443411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.444072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.445973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.450115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e740-41ad-428f-a6e1-04c9f89ad5ef admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.455940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.456543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.456823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.457496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.458148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.460008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.476046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 962/1964] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:35.476198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1003/1965] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:35.476845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.480970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.487963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a247c2-cf84-4f3b-8153-03c5d4afb025 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.547877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.548560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.548824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.549126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] PATH_INFO: `/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370/users/469f3f79bd2e47bb8f5e16f29e44560c/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.551094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27f87b3ab7c4867aa7f3b84c5e21370, user_id=469f3f79bd2e47bb8f5e16f29e44560c, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.551558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd59a7-28ec-4eff-a615-4d63d5aa932f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.553961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb828a-bb97-46ae-9f28-97c4ea07bacb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.557648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.558267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.558549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.558824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.559445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.561400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.567188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 963/1966] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:35 2022] PUT /identity/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370/users/469f3f79bd2e47bb8f5e16f29e44560c/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:35.580231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.581269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.581445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.585010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1004/1967] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:35.589189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.600037 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f38415-7c1d-4c46-899d-de0dadfcc2b4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.615306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] MFA Rules not processed for user `469f3f79bd2e47bb8f5e16f29e44560c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:35.616576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.689182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.689976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.690255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.690524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.691401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.693970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.703260 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 964/1968] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:35.706329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1005/1969] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:35.711494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.725954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0b98-874e-448d-9737-2dc38f3fdd2d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.733000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.733668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.734112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.734394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.735035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.736996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.737467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.739720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.749387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 965/1970] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:35.753283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:35.775002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7afef-297c-4c03-ac14-00cfccd32548 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.775192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:35.775379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.775379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.775379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.775379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] PATH_INFO: `/v3/projects/690868d14d504b8e911687e57898a03c/users/d0e319009bad4882a7eb40793b7bed13/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.789328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690868d14d504b8e911687e57898a03c, user_id=d0e319009bad4882a7eb40793b7bed13, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.792077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d164ac-feb4-44da-8a76-bfdc844746e5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.809366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b461129-19ee-4e8b-a51a-c32f7ecb7beb admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:35.809888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 966/1971] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:35 2022] PUT /identity/v3/projects/690868d14d504b8e911687e57898a03c/users/d0e319009bad4882a7eb40793b7bed13/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:35.821699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.822428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.822739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.847622 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:35.859997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] MFA Rules not processed for user `d0e319009bad4882a7eb40793b7bed13`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:35.861238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:35.867483 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:35.869039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:35.869515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:35.869923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:35.938704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 967/1972] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:35.949846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:35.951304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:35.963094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1006/1973] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:35 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:36.006186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:36.016340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce44512-4b9e-4cc3-acdf-5c672920fecd None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:36.020213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:36.020834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:36.021174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:36.021497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:36.033010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:36.035097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:36.052488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 968/1974] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:36.222594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:36.224104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:36.244182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4317-68ae-4ab5-b5af-418c098be026 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:36.249010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:36.249661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:36.249936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:36.250299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:36.252305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:36.328230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:36.329934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:36.341398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1007/1975] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:36.403604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:36.414397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65050a-f740-47fd-bef1-f8d59e786b48 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:36.422050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:36.422674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:36.422956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:36.423231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:36.428362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:36.545235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:36.553600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82531202-4daa-48d9-9af3-a4ee494c5891 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:36.574466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 969/1976] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:36 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:38.523341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:38.525323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:38.551112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:38.589826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84243f8-13d7-42bc-883c-437caf533874 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:38.594825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:38.595463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:38.595784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:38.596057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] PATH_INFO: `/v3/users/4b9bdaf285254851a56834d1cacd9502` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:38.597185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9bdaf285254851a56834d1cacd9502)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:38.599110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:38.647390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9bdaf285254851a56834d1cacd9502'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:38.648088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9bdaf285254851a56834d1cacd9502'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:38.656078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9bdaf285254851a56834d1cacd9502'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:38.663600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9bdaf285254851a56834d1cacd9502'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:38.676610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9bdaf285254851a56834d1cacd9502'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:38.682560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1008/1977] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:38 2022] DELETE /identity/v3/users/4b9bdaf285254851a56834d1cacd9502 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:38.687082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:38.687528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:38.687687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:38.708284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:38.709505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:38.770205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 970/1978] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:38.780356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:38.781746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:38.844189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66614cbb-6310-404c-abf4-738a6436ef4e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:38.848573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:38.849184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:38.849530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:38.849723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:38.859568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:38.861019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:38.871540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1009/1979] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:39.201163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:39.203312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:39.216774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11004e-912f-4ee5-beb6-c80a67bcc0a6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:39.224015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:39.224779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:39.225133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:39.225480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] PATH_INFO: `/v3/projects/c759b16c6bf64e529122be5aa0825961` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:39.226731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c759b16c6bf64e529122be5aa0825961)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:39.229649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:39.271054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:39.321928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:39.323909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:39.347999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e9746-835b-4a39-aaae-7e544969eed6 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:39.350854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c759b16c6bf64e529122be5aa0825961', 'request_id': 'req-b0cbf310-253a-4382-9747-f063df35707f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.360959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c759b16c6bf64e529122be5aa0825961', 'request_id': 'req-b0cbf310-253a-4382-9747-f063df35707f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.372935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c759b16c6bf64e529122be5aa0825961', 'request_id': 'req-b0cbf310-253a-4382-9747-f063df35707f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.395846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] The token cache is being invalidate because project c759b16c6bf64e529122be5aa0825961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:39.396513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.397074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.399637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 971/1980] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:39 2022] DELETE /identity/v3/projects/c759b16c6bf64e529122be5aa0825961 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:39.403342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:39.405018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:39.423790 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:39.457900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:39.459817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:39.460490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:39.460758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] PATH_INFO: `/v3/users/d370befd3b844a0d833305b932702574` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:39.463229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d370befd3b844a0d833305b932702574)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:39.465381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:39.534792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd370befd3b844a0d833305b932702574'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.535463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd370befd3b844a0d833305b932702574'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.546752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd370befd3b844a0d833305b932702574'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.548735 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbf310-253a-4382-9747-f063df35707f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:39.558499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd370befd3b844a0d833305b932702574'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.564976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:39.565813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:39.566581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:39.566783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] PATH_INFO: `/v3/users/f76522a55afd4e5c914684391b2ff6ad` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:39.568224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76522a55afd4e5c914684391b2ff6ad)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:39.570651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:39.577258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92a737f7-f74a-42dd-b5fc-ce8c06c00790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd370befd3b844a0d833305b932702574'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.583999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1010/1981] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:39 2022] DELETE /identity/v3/users/d370befd3b844a0d833305b932702574 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:39.588817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:39.589275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:39.589553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:39.609342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:39.610552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:39.646397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76522a55afd4e5c914684391b2ff6ad'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.647290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76522a55afd4e5c914684391b2ff6ad'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.656266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76522a55afd4e5c914684391b2ff6ad'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.668727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76522a55afd4e5c914684391b2ff6ad'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.677354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1011/1982] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:39.677523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76522a55afd4e5c914684391b2ff6ad'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:39.682949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 972/1983] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:39 2022] DELETE /identity/v3/users/f76522a55afd4e5c914684391b2ff6ad => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:39.685848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:39.686309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:39.745594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e404445-a5d6-4188-9eb1-1a302ffac66d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:39.764572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:39.809914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:39.810815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:39.811247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:39.811719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:39.903066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:39.905363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d38e3-602d-4f3c-8d2a-96e630ee62c1 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:39.918502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1012/1984] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:40.182474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.183970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.196832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5221dc-126b-4698-ac09-8808f628e892 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.203461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.204391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.204770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.205132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] PATH_INFO: `/v3/projects/c61282fba32f4599814b39d37261de53` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.206394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61282fba32f4599814b39d37261de53)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.208803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.236790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:40.265280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61282fba32f4599814b39d37261de53', 'request_id': 'req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.271129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61282fba32f4599814b39d37261de53', 'request_id': 'req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.281417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61282fba32f4599814b39d37261de53', 'request_id': 'req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.293520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] The token cache is being invalidate because project c61282fba32f4599814b39d37261de53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:40.293810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.294585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.296168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 973/1985] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:40 2022] DELETE /identity/v3/projects/c61282fba32f4599814b39d37261de53 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:40.301117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477ce4-2c69-40cd-b782-9f81ce7e441b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.301342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477ce4-2c69-40cd-b782-9f81ce7e441b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.301620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb477ce4-2c69-40cd-b782-9f81ce7e441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.321812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-fb477ce4-2c69-40cd-b782-9f81ce7e441b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:40.323043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-fb477ce4-2c69-40cd-b782-9f81ce7e441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.344972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.347008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.407151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1013/1986] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:40.415296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.415617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.415893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.425443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37002-601c-46b3-8dd6-c6a4ce3da1b7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.430051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.430702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.430976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.431239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] PATH_INFO: `/v3/projects/edb747bf9ec049eb9a2928f7b37edc0b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.432375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb747bf9ec049eb9a2928f7b37edc0b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.434277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.434775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:40.436014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.445123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1014/1987] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:40.449206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.458429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa90c7-65aa-4bb1-976d-e60590c623b2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.463965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.464586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.464902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.465204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.465849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:40.466146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.470518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.497838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1015/1988] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:40.498006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb747bf9ec049eb9a2928f7b37edc0b', 'request_id': 'req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.499096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb747bf9ec049eb9a2928f7b37edc0b', 'request_id': 'req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.500505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.509201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb747bf9ec049eb9a2928f7b37edc0b', 'request_id': 'req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.513683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d65-7986-461b-b2fa-599b2b8bef74 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.529149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] The token cache is being invalidate because project edb747bf9ec049eb9a2928f7b37edc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:40.529149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.529636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.531650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 974/1989] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:40 2022] DELETE /identity/v3/projects/edb747bf9ec049eb9a2928f7b37edc0b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:40.535359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.536786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.579337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:40.593046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.593846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.594133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.594851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.595568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.597913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.614388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:40.641696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6aa2-e6f5-47e3-aa20-5c2915fc8d65 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.653891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1016/1990] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:40 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:40.657617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.659420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.660203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.660786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.661066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.661390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] PATH_INFO: `/v3/users/0ed3ed09ecb9476d82ff9e7e6ee71dbc` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.662719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed3ed09ecb9476d82ff9e7e6ee71dbc)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.665710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.719689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed3ed09ecb9476d82ff9e7e6ee71dbc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.720510 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed3ed09ecb9476d82ff9e7e6ee71dbc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.732601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed3ed09ecb9476d82ff9e7e6ee71dbc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.737945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e856dd3-3662-49a8-a03a-1049c8406829 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.743028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.743675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.743953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.744218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.744946 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.746938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.750145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed3ed09ecb9476d82ff9e7e6ee71dbc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.759957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed3ed09ecb9476d82ff9e7e6ee71dbc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:40.766408 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 975/1991] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:40 2022] DELETE /identity/v3/users/0ed3ed09ecb9476d82ff9e7e6ee71dbc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:40.770366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1017/1992] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:40 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:40.774743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.776247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:40.847886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a661c-6e14-4584-8c23-0bfcf4aba572 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.853083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.853681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.853995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.854271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.854923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.856899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.868057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 976/1993] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:40.871868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.885624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd678-001a-4226-95b5-ec1837c71829 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.890214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.890854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.891128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.891395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.892080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.893670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc127-9650-482c-a963-247a71c3ab44 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.904878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1018/1994] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:40.909016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:40.919421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0a2c7-cc20-404b-8b40-0cf78b7afdc8 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:40.924339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:40.925004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.925284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.925563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] PATH_INFO: `/v3/projects/f7ca6b52c2b84d909ae0283019afe298/users/469cd3eccab4477e8e640e5a9b310872/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:40.941406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ca6b52c2b84d909ae0283019afe298, user_id=469cd3eccab4477e8e640e5a9b310872, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:40.944481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:40.959497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 977/1995] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:40 2022] PUT /identity/v3/projects/f7ca6b52c2b84d909ae0283019afe298/users/469cd3eccab4477e8e640e5a9b310872/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:40.970824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:40.971223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:40.971556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.000581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] MFA Rules not processed for user `469cd3eccab4477e8e640e5a9b310872`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:41.001840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.077493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.079692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1019/1996] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:41.084045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.084184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.129887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:41.152620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:41.174222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d05b0-7bf8-4c6c-b7b6-7db92bbf712f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.174894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f619b-d9d9-4f80-b3e4-44d215240fe1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.181359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.181560 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.182224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.182435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.182685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.183131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.183238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] PATH_INFO: `/v3/projects/4d5d57a733ea4a4b88838cbe7c7ff6b3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.183324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.184062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.184336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5d57a733ea4a4b88838cbe7c7ff6b3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.186230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.186524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.218126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1020/1997] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:41.224781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.244158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5d57a733ea4a4b88838cbe7c7ff6b3', 'request_id': 'req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.251332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5d57a733ea4a4b88838cbe7c7ff6b3', 'request_id': 'req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.262055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5d57a733ea4a4b88838cbe7c7ff6b3', 'request_id': 'req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.276485 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] The token cache is being invalidate because project 4d5d57a733ea4a4b88838cbe7c7ff6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:41.276784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.277576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.280603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 978/1998] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:41 2022] DELETE /identity/v3/projects/4d5d57a733ea4a4b88838cbe7c7ff6b3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:41.291093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.292907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.295558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288a63-5b27-470e-bb99-cd269ec2de9a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.356070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.357246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.357530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.357801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.358423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.358602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0d8ac-56da-4f08-8cb3-25dc11ae9dfb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.360382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.364533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.365198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.365480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.365755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] PATH_INFO: `/v3/users/50132f84fee3458b9af4c70d4e136a51` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.366899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50132f84fee3458b9af4c70d4e136a51)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.369873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.383031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1021/1999] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:41.387778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.390224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.404117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b82b3-92db-469f-9a5a-cbbf0fe77944 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.409285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.410016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.410375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.410733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.411456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.413320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.421023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50132f84fee3458b9af4c70d4e136a51'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.421843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50132f84fee3458b9af4c70d4e136a51'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.428801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1022/2000] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:41.433156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.434923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50132f84fee3458b9af4c70d4e136a51'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.435066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.445771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:41.449219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abd72d-2c19-4e55-8d03-83857606ae34 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.450737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50132f84fee3458b9af4c70d4e136a51'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.454340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.454856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.455143 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.455397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.456068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.457730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.466450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0e5598ea-543a-4ceb-87e3-404615ad6ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50132f84fee3458b9af4c70d4e136a51'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:41.473828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1023/2001] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:41.478566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.483310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 979/2002] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:41 2022] DELETE /identity/v3/users/50132f84fee3458b9af4c70d4e136a51 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:41.489213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a3d5e-0803-4844-a8ed-c204fdd88fac admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.539284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:41.557706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.558346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.558620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.558892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] PATH_INFO: `/v3/projects/0c72b4f38f504d1ea97a2411762be365/users/2f2167d384a94770996ae3cc78d48fe3/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.575287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c72b4f38f504d1ea97a2411762be365, user_id=2f2167d384a94770996ae3cc78d48fe3, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.578045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.591839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1024/2003] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:41 2022] PUT /identity/v3/projects/0c72b4f38f504d1ea97a2411762be365/users/2f2167d384a94770996ae3cc78d48fe3/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:41.603925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.604232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.604539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.635379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] MFA Rules not processed for user `2f2167d384a94770996ae3cc78d48fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:41.636654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.719523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 980/2004] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:41.721311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.785122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.786601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:41.824794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16af6b4-a2e1-4a30-b4cb-c72742c3ad13 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.833890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.834503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.835013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.835352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.836115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.846630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.884755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1025/2005] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:41.888835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:41.899244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80413f-9ddb-41ee-bf84-20bc0a953a95 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:41.986526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:41.987182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:41.987459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:41.987760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] PATH_INFO: `/v3/projects/f9fed42d80004fc6bdbe894238591c84` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:41.988911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9fed42d80004fc6bdbe894238591c84)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:41.990825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:41.990938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76265f21-29f2-4171-9a4b-97951dd891cd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.000245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.000867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.001149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.001420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.002054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.004007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.018144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:42.047467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1026/2006] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:41 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:42.052460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.052919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9fed42d80004fc6bdbe894238591c84', 'request_id': 'req-9c82f921-f91e-44fa-87d2-6743fb1583b6'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:42.059446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9fed42d80004fc6bdbe894238591c84', 'request_id': 'req-9c82f921-f91e-44fa-87d2-6743fb1583b6'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:42.064893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc94c-2d84-413b-ab6e-117265d2f516 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.067453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9fed42d80004fc6bdbe894238591c84', 'request_id': 'req-9c82f921-f91e-44fa-87d2-6743fb1583b6'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:42.082282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] The token cache is being invalidate because project f9fed42d80004fc6bdbe894238591c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:42.082574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:42.083262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9c82f921-f91e-44fa-87d2-6743fb1583b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:42.089994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 981/2007] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:41 2022] DELETE /identity/v3/projects/f9fed42d80004fc6bdbe894238591c84 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:42.092137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841583f8-28a5-430c-ab86-8b693652ac59 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.092422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841583f8-28a5-430c-ab86-8b693652ac59 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.092696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841583f8-28a5-430c-ab86-8b693652ac59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.114846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:42.115124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-841583f8-28a5-430c-ab86-8b693652ac59 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:42.119603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-841583f8-28a5-430c-ab86-8b693652ac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.133754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.134411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.134692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.135013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.135653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.137749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.150871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1027/2008] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:42.151921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.153678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.166154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6359b-0217-4caf-89a2-41e1166ff315 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.170981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.171652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.171931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.172203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] PATH_INFO: `/v3/projects/a5c9d451d3934663b634ae2aca7c1a39/users/69a1bfdd57b14f8788b43d34e3b92edb/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.181018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 982/2009] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:42.188576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c9d451d3934663b634ae2aca7c1a39, user_id=69a1bfdd57b14f8788b43d34e3b92edb, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.189269 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.189701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.189817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.191285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.204643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1028/2010] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:42 2022] PUT /identity/v3/projects/a5c9d451d3934663b634ae2aca7c1a39/users/69a1bfdd57b14f8788b43d34e3b92edb/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:42.208416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.210181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:42.211415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.270978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c8f37-a18f-43f5-981d-25154e9b422a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.275859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.276398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.276670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.276940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.277571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.281736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 983/2011] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:42.281886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.283035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.292238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1029/2012] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:42.296636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.297711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e78e9-998d-4b03-bb28-19555cfc92ea None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.302731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.304287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.304410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.304949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.306846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:42.307157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.307703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e1e39-b6dd-4a50-8bfc-e6800f8f2c60 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.308888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.312910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.313626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.313900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.314163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] PATH_INFO: `/v3/projects/a5c9d451d3934663b634ae2aca7c1a39/users/69a1bfdd57b14f8788b43d34e3b92edb/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.316324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c9d451d3934663b634ae2aca7c1a39, user_id=69a1bfdd57b14f8788b43d34e3b92edb, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.319354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7603-60a7-43ea-b1ad-9d23a0eb6030 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.321172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 984/2013] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:48:42.325259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.334783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1030/2014] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:42 2022] PUT /identity/v3/projects/a5c9d451d3934663b634ae2aca7c1a39/users/69a1bfdd57b14f8788b43d34e3b92edb/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:42.336324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc9de5-6b2f-4338-81c4-686ec1d4645f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.345541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.345950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.346233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.375305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:42.381618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] MFA Rules not processed for user `69a1bfdd57b14f8788b43d34e3b92edb`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:42.383270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.404982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.405859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.406244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.406624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.407358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.410069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.446895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 985/2015] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:42.448114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.459117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1031/2016] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:42.484042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.485670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.514082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:42.526851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b1ffd-5f8a-4b17-9967-1d34459f25bb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.532928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.533596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.536477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.536784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.537479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.540348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.553119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13118cad-ab2b-4b6d-8c6f-7a368c298af0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.567513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 986/2017] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:42.569597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.579675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143ec4a-e65f-4972-9a6d-be0d4afef0d5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.584631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.585530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.585915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.586322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.587185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.589043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.599567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 987/2018] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:42.604738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.608162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.608778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.609054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.609324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.611129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.614782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cb2-0224-4027-b133-812ac08f5e0d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.620116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.620711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.620990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.621268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.621933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.623556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.634563 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 988/2019] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:42.638853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.650203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f6d9-b268-4679-8ade-b8fade8ec27a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.655352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.656004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.656290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.656564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] PATH_INFO: `/v3/projects/ec3b942a44534e12a7553a6a7aff41fe/users/c60f07da0dd24502b153e9bae19dcf63/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.672290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3b942a44534e12a7553a6a7aff41fe, user_id=c60f07da0dd24502b153e9bae19dcf63, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.674996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.676261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.677835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fb4c4-a6db-4ac1-b452-43916c731c9a service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.688827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1032/2020] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:42.689698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 989/2021] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:42 2022] PUT /identity/v3/projects/ec3b942a44534e12a7553a6a7aff41fe/users/c60f07da0dd24502b153e9bae19dcf63/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:42.702512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.702795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.703069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.732096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] MFA Rules not processed for user `c60f07da0dd24502b153e9bae19dcf63`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:42.733330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:42.800422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1033/2022] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:42.805022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.872869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c6a6b-574a-4be2-866a-d6b44af2f26a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.877971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:42.878483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:42.878767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:42.879041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:42.879726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:42.881717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:42.894904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:42.915663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 990/2023] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:42.919787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:42.933161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575e47e-2ba7-4bd2-b39b-215aba9050f4 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:42.956460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:43.017307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.017978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.018350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.018803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.019452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.021795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.051720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1034/2024] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:42 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:43.057779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.069493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676585e7-c759-4e9d-8ef3-5b517a58dd80 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.075617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.076393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.076910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.077344 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.078139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.081039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.094480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 991/2025] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:43.098786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.116878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f598d45-8bb2-45ca-a89f-b65220a71941 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.125658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.126445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.126792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.127133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] PATH_INFO: `/v3/projects/4d2c1c8d9d424125baa5a23b03987542/users/80fb2b8f26ca418b9548396a4f1bdb65/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.163442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2c1c8d9d424125baa5a23b03987542, user_id=80fb2b8f26ca418b9548396a4f1bdb65, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.166839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.190112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1035/2026] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:43 2022] PUT /identity/v3/projects/4d2c1c8d9d424125baa5a23b03987542/users/80fb2b8f26ca418b9548396a4f1bdb65/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:43.191152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.202723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37e7e-a4eb-4523-9287-e682bb853171 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.262591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.263198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.263507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.263778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.264677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.266350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87133d6-e717-4198-b6bb-ea5908d1c262 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.276299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 992/2027] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:43.280549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.291283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20d613-2a33-49e3-a9f6-b2ea7b8a3c62 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.296169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.296838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.297117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.297417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] PATH_INFO: `/v3/projects/4d2c1c8d9d424125baa5a23b03987542/users/80fb2b8f26ca418b9548396a4f1bdb65/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.299385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2c1c8d9d424125baa5a23b03987542, user_id=80fb2b8f26ca418b9548396a4f1bdb65, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.302100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.313301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1036/2028] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:43 2022] PUT /identity/v3/projects/4d2c1c8d9d424125baa5a23b03987542/users/80fb2b8f26ca418b9548396a4f1bdb65/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:43.323386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.323824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.324120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.353346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] MFA Rules not processed for user `80fb2b8f26ca418b9548396a4f1bdb65`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:43.354618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:43.422219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 993/2029] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:43.433629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.489270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f4589-b7df-49e5-937a-e9c823ce7e6e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.493054 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.493661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.493939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.494209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.503541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.504967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.514342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1037/2030] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:43.620392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:43.630195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42e5bc-40bb-4aa2-97ce-aa0fb7808fc7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:43.633881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:43.634486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:43.634904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:43.635201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:43.695546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:43.696902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b36fa-819a-48f1-9c52-d0c252977f10 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:43.701376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:43.706179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 994/2031] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:44.609834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:44.611277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:44.691630 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa950cf-7987-4a35-90c3-f03819109954 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:44.697503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:44.698146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:44.698447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:44.698704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] PATH_INFO: `/v3/users/d0e319009bad4882a7eb40793b7bed13` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:44.699868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e319009bad4882a7eb40793b7bed13)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:44.701786 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:44.724875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:44.757550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e319009bad4882a7eb40793b7bed13'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:44.758252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e319009bad4882a7eb40793b7bed13'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:44.766712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e319009bad4882a7eb40793b7bed13'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:44.775758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e319009bad4882a7eb40793b7bed13'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:44.790667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e319009bad4882a7eb40793b7bed13'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:44.798546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1038/2032] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:44 2022] DELETE /identity/v3/users/d0e319009bad4882a7eb40793b7bed13 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:44.803182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:44.803673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:44.803867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:44.825511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:44.826747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:44.891410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 995/2033] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:44.901878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:44.903345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:44.963788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4988-cf0a-478a-8964-9a922af950c8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:44.967897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:44.968798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:44.969073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:44.969345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:44.979342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:44.980829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa24c70-1a97-4d26-900e-7512991128ed service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:44.990712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1039/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:45.291470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.292993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:45.312661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d398a-4e48-4fe0-b448-1c28db5fdafe None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.321969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.325320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.325766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.326213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] PATH_INFO: `/v3/projects/690868d14d504b8e911687e57898a03c` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.327804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=690868d14d504b8e911687e57898a03c)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.330525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.426099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690868d14d504b8e911687e57898a03c', 'request_id': 'req-896fecb8-6c94-4394-9ce4-827f2b8de9cf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:45.439633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690868d14d504b8e911687e57898a03c', 'request_id': 'req-896fecb8-6c94-4394-9ce4-827f2b8de9cf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:45.464697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690868d14d504b8e911687e57898a03c', 'request_id': 'req-896fecb8-6c94-4394-9ce4-827f2b8de9cf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:45.488571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] The token cache is being invalidate because project 690868d14d504b8e911687e57898a03c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:45.490814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:45.492609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:45.501154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 996/2035] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:45 2022] DELETE /identity/v3/projects/690868d14d504b8e911687e57898a03c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:45.503080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0abd-43e4-42d4-bf93-14cf262de35b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.503368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0abd-43e4-42d4-bf93-14cf262de35b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.503678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc0abd-43e4-42d4-bf93-14cf262de35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.547224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a8bc0abd-43e4-42d4-bf93-14cf262de35b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:45.548517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a8bc0abd-43e4-42d4-bf93-14cf262de35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:45.612626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.614555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:45.663363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1040/2036] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:45.671425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.671826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.672033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.688969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fecb8-6c94-4394-9ce4-827f2b8de9cf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.692609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.693216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.693491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.693760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.694368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:45.695548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:45.696144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:45.704358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1041/2037] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:45.708504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.718184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa939-aa71-4da7-99fb-a5ab37fd1088 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.723792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.724392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.724668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.724935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.725963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:45.726423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.728085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.737708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:45.740109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1042/2038] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:48:45.745655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.756021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e5d6b-1245-4674-bb3d-8d406132275b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.761054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.761917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.762504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.762615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.762785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.763047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.763678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.765544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.771739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 997/2039] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:45.785884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:45.794839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1043/2040] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:45 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:45.798173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.809672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd277b4-f2b7-4118-a9ac-8c93d41860b1 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.868960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.869615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.869857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.870124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.871396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.872845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.902369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 998/2041] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:45 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:45.906551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.919503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b05ba-a255-4220-9fd2-13c1a6e79b17 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.926119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.926933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.927401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.928059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.929032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.931399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.945198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1044/2042] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:45.947309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:45.961355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7cef7-4723-41ca-87c9-430f9c7f522c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:45.967973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:45.969517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:45.969803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:45.970075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:45.971065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:45.972716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf02a35-97c6-4484-b40a-1800ee3960a1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:45.990581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 999/2043] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:45.991826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:46.002669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0be2-7e80-44f5-b7d2-197c83a0485c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:46.014378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:46.015266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.015571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.015970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] PATH_INFO: `/v3/projects/7e1158cd7f24450abfa06ee83a159c7d/users/0f1b612f1d3b45748c063509c48f65a0/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.041060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1158cd7f24450abfa06ee83a159c7d, user_id=0f1b612f1d3b45748c063509c48f65a0, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:46.043957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:46.065411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1045/2044] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:45 2022] PUT /identity/v3/projects/7e1158cd7f24450abfa06ee83a159c7d/users/0f1b612f1d3b45748c063509c48f65a0/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:46.087429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52398144-149d-4477-b4bc-bf1ccd4a761f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.087429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52398144-149d-4477-b4bc-bf1ccd4a761f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.087429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52398144-149d-4477-b4bc-bf1ccd4a761f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.125430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-52398144-149d-4477-b4bc-bf1ccd4a761f None None] MFA Rules not processed for user `0f1b612f1d3b45748c063509c48f65a0`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:46.127775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-52398144-149d-4477-b4bc-bf1ccd4a761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:46.205209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1000/2045] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:46.216995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:46.219146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:46.238038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c857-f074-4592-bcaa-e683cc42aa13 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.238609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c857-f074-4592-bcaa-e683cc42aa13 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.238858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38c857-f074-4592-bcaa-e683cc42aa13 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.240486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1001/2046] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:48:46.252620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.252918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.253305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.276457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:46.277961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:46.318153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0b35-cabe-46c5-8c4b-420605057845 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:46.322732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:46.323577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.323862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.324557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.329258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:46.334070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:46.335564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:46.345990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1046/2047] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:46.372028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1002/2048] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:48:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:46.390007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:46.391612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:46.412967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365b8a-88c9-42fd-a10d-97e622de6a93 service glance] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:46.419714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:46.420416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:46.420596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:46.420868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:46.430613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:46.432352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:46.443947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1047/2049] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:47.064176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:47.065731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:47.129659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cda2a5-535b-460a-a1b0-272f3980b8bf None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:47.134128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:47.134841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:47.135188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:47.135565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:47.138227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:47.227121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:47.228913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3115c13-e7c7-46c4-8b91-64acba68c179 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:47.242149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1003/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:49.647194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:49.649222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:49.766182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae39a4-14ae-4350-8ebc-c021dbc744bc service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:49.771452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:49.772133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:49.772409 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:49.772680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] PATH_INFO: `/v3/users/4e27317d68904a50b30eb5e8d7ac9f71` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:49.774367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e27317d68904a50b30eb5e8d7ac9f71)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:49.776343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:49.840555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:49.857738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e27317d68904a50b30eb5e8d7ac9f71'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:49.858469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e27317d68904a50b30eb5e8d7ac9f71'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:49.866493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e27317d68904a50b30eb5e8d7ac9f71'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:49.874108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e27317d68904a50b30eb5e8d7ac9f71'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:49.888707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e27317d68904a50b30eb5e8d7ac9f71'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:49.897546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1048/2051] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:49 2022] DELETE /identity/v3/users/4e27317d68904a50b30eb5e8d7ac9f71 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:49.904908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:49.905453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:49.905551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:49.925209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:49.926695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:49.985163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1004/2052] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:49.995020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:49.996472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:50.059344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67c01-91b3-4165-bb16-9a5b0dcd0e25 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:50.064337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:50.065005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:50.065292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:50.065604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:50.077787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:50.079396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5265a4-e4e2-46a6-b3d2-85ade43d9fba service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:50.091634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1049/2053] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:50.171023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:50.172591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:50.185567 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1719fb-1997-42ad-9bec-0dc7725c486d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:50.190312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:50.190969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:50.191245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:50.191541 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] PATH_INFO: `/v3/users/c60f07da0dd24502b153e9bae19dcf63` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:50.197559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60f07da0dd24502b153e9bae19dcf63)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:50.199616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:50.225446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:50.256250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60f07da0dd24502b153e9bae19dcf63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.256949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60f07da0dd24502b153e9bae19dcf63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.264980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60f07da0dd24502b153e9bae19dcf63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.278431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60f07da0dd24502b153e9bae19dcf63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.286360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60f07da0dd24502b153e9bae19dcf63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.293887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1005/2054] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:50 2022] DELETE /identity/v3/users/c60f07da0dd24502b153e9bae19dcf63 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:50.303211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:50.303535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:50.303862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:50.325793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:50.327272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:50.412587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1050/2055] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:50.429120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:50.437359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:50.531679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a0eb0-0fbf-4588-baee-70a4ea38f45d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:50.541674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:50.542376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:50.542655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:50.542911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:50.555595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:50.557767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:50.580892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1006/2056] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:50.774987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:50.776624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:50.789324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756d765-f789-43cb-977b-05bc8531a142 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:50.793956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:50.794605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:50.794885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:50.795160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] PATH_INFO: `/v3/projects/1456c3fed4db4882aa65f6066ae32725` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:50.796624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1456c3fed4db4882aa65f6066ae32725)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:50.798670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:50.858374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1456c3fed4db4882aa65f6066ae32725', 'request_id': 'req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.860322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:50.876404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1456c3fed4db4882aa65f6066ae32725', 'request_id': 'req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.895664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1456c3fed4db4882aa65f6066ae32725', 'request_id': 'req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.914148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] The token cache is being invalidate because project 1456c3fed4db4882aa65f6066ae32725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:50.915231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.916039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:50.919532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1051/2057] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:50 2022] DELETE /identity/v3/projects/1456c3fed4db4882aa65f6066ae32725 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:50.922048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:50.935773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16b1af-1460-4a31-8d3b-a0cad4b91ebd service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:50.997532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:50.999062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:51.118717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:51.119511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:51.119806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:51.120054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] PATH_INFO: `/v3/users/469f3f79bd2e47bb8f5e16f29e44560c` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:51.121971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469f3f79bd2e47bb8f5e16f29e44560c)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:51.125117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:51.133801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c8610-0a67-4f3b-98a8-ef2368e74c4b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:51.145752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:51.146526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:51.146855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:51.147121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] PATH_INFO: `/v3/projects/ec3b942a44534e12a7553a6a7aff41fe` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:51.149249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3b942a44534e12a7553a6a7aff41fe)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:51.152048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:51.233689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:51.235689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469f3f79bd2e47bb8f5e16f29e44560c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.236588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469f3f79bd2e47bb8f5e16f29e44560c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.248492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469f3f79bd2e47bb8f5e16f29e44560c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.272898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3b942a44534e12a7553a6a7aff41fe', 'request_id': 'req-78e84240-bf9a-4f5e-824b-c276dd57ffb1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.278935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469f3f79bd2e47bb8f5e16f29e44560c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.280908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3b942a44534e12a7553a6a7aff41fe', 'request_id': 'req-78e84240-bf9a-4f5e-824b-c276dd57ffb1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.288534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469f3f79bd2e47bb8f5e16f29e44560c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.291531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3b942a44534e12a7553a6a7aff41fe', 'request_id': 'req-78e84240-bf9a-4f5e-824b-c276dd57ffb1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.299204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1007/2058] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:50 2022] DELETE /identity/v3/users/469f3f79bd2e47bb8f5e16f29e44560c => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:51.306556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] The token cache is being invalidate because project ec3b942a44534e12a7553a6a7aff41fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:51.306556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.307207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.309206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1052/2059] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:50 2022] DELETE /identity/v3/projects/ec3b942a44534e12a7553a6a7aff41fe => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:51.313135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:51.314600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:51.438770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492faa2-4b08-4e6f-a563-3dac39f1f425 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:51.445973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:51.446547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:51.447178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:51.447178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] PATH_INFO: `/v3/users/80fb2b8f26ca418b9548396a4f1bdb65` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:51.448289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80fb2b8f26ca418b9548396a4f1bdb65)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:51.450246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:51.545031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80fb2b8f26ca418b9548396a4f1bdb65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.546003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80fb2b8f26ca418b9548396a4f1bdb65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.572029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80fb2b8f26ca418b9548396a4f1bdb65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.601257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80fb2b8f26ca418b9548396a4f1bdb65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.609370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80fb2b8f26ca418b9548396a4f1bdb65'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:51.616929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1008/2060] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:51 2022] DELETE /identity/v3/users/80fb2b8f26ca418b9548396a4f1bdb65 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:51.783802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:51.787608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:51.886567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84240-bf9a-4f5e-824b-c276dd57ffb1 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:51.900917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:51.950712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:51.951347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:51.951652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:51.951921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] PATH_INFO: `/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:51.953223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27f87b3ab7c4867aa7f3b84c5e21370)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:51.955069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.009783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27f87b3ab7c4867aa7f3b84c5e21370', 'request_id': 'req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.016443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27f87b3ab7c4867aa7f3b84c5e21370', 'request_id': 'req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.022233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.024883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.029712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27f87b3ab7c4867aa7f3b84c5e21370', 'request_id': 'req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.043691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688fb8f-3909-4ecc-b8f7-28da553cf790 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.045380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] The token cache is being invalidate because project d27f87b3ab7c4867aa7f3b84c5e21370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:52.046015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.046825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5099e4bf-175c-4c5f-bb5a-3db199c1e38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.048976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1053/2061] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:51 2022] DELETE /identity/v3/projects/d27f87b3ab7c4867aa7f3b84c5e21370 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:52.053805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b89dd-9cf0-4003-a98e-78760ba4cc18 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.054286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b89dd-9cf0-4003-a98e-78760ba4cc18 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.054594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b89dd-9cf0-4003-a98e-78760ba4cc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.077323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1b8b89dd-9cf0-4003-a98e-78760ba4cc18 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.078645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b8b89dd-9cf0-4003-a98e-78760ba4cc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.105439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:52.128581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.129236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.129514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.129790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] PATH_INFO: `/v3/projects/4d2c1c8d9d424125baa5a23b03987542` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.130925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2c1c8d9d424125baa5a23b03987542)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.133776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.145200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1054/2062] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:52.153678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.154073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.154356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.175179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.176450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.188834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2c1c8d9d424125baa5a23b03987542', 'request_id': 'req-f9de3631-6790-48cf-bfa5-c1078ff76e85'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.196284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2c1c8d9d424125baa5a23b03987542', 'request_id': 'req-f9de3631-6790-48cf-bfa5-c1078ff76e85'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.204168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2c1c8d9d424125baa5a23b03987542', 'request_id': 'req-f9de3631-6790-48cf-bfa5-c1078ff76e85'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.215797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] The token cache is being invalidate because project 4d2c1c8d9d424125baa5a23b03987542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:52.216116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.216751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f9de3631-6790-48cf-bfa5-c1078ff76e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:52.218249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1009/2063] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:52 2022] DELETE /identity/v3/projects/4d2c1c8d9d424125baa5a23b03987542 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:52.221429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:52.223186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429c1c2-1d9f-4234-8c93-d2f41e0e4471 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.223468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429c1c2-1d9f-4234-8c93-d2f41e0e4471 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.223819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429c1c2-1d9f-4234-8c93-d2f41e0e4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.241521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1055/2064] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:52.243015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9429c1c2-1d9f-4234-8c93-d2f41e0e4471 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.244273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9429c1c2-1d9f-4234-8c93-d2f41e0e4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.245900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.255028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1010/2065] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:52.255216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac014372-960c-403c-8b52-0eb19142be73 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.260329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.261161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.261434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.261699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.262336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:52.262436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.262633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.262746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.262951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.264309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.282050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1056/2066] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:48:52.284419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.284727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.285939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.294348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1011/2067] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:52.295789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ead3a-6bc3-4e8f-a732-66cb762ab16e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.298420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.301653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.302268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.302540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.302824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.303437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.305430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.309727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b796648-f290-49f4-a2cd-847fd74498f7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.318226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.318848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.319438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.319438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.320049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:52.320944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.322581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.341116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1057/2068] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:52.348400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1012/2069] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:48:52.348542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.349861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.359600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed28e30-2ec5-4320-95cf-9d39124476d8 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.411938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df525e-6c7b-4bc2-9bab-0f0f829bd0dd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.415966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.416579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.416768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.416894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.417108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.417356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.417636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.417751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.417913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.418508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.419738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.420466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.442813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1058/2070] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:52.447125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1013/2071] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:52.447425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.450975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.459863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710f0ac-3294-4d49-aa0a-fdce2a9b12b8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.482998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:52.514055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a112-56bb-4301-9cf0-ed1cc678c131 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.517984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.518584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.518950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.519089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.519285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.519592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.519873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.520065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.520188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.520754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.521647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.522662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.533076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1059/2072] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:52.537399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.546954 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1014/2073] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:48:52.550071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5bd2f-0c4a-4de3-a2c4-3271f6ef286b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.551006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.553061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.553654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.553931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.554199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.555363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.557185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.561267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01a5ce-cb4e-4417-a1fa-2b98ef46f25f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.565948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.566535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.566810 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.567077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.567733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.568403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1060/2074] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:52.572527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.572988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.585919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1015/2075] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:52.586085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff306a1-6752-43b1-8e49-688c0e1f1b69 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.586903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.588080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.588727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.589006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.589274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] PATH_INFO: `/v3/projects/f78aeb95f9714bac9460428653cfe214/users/f86173a3c86d43d9942808fb2a3df896/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.596783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8f710-4c67-402d-8eae-65c062653bfb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.601710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.602356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.602638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.602905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.603570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.605160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.607891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78aeb95f9714bac9460428653cfe214, user_id=f86173a3c86d43d9942808fb2a3df896, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.611339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e0d12-7e9f-4652-a0dc-dfa3fd011fe7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.620114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1016/2076] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:52.621226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.626716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1061/2077] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:52 2022] PUT /identity/v3/projects/f78aeb95f9714bac9460428653cfe214/users/f86173a3c86d43d9942808fb2a3df896/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:52.634190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa600e-5a71-4650-9102-f1042ad9107d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.638730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.639092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.639371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.651856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:52.674249 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] MFA Rules not processed for user `f86173a3c86d43d9942808fb2a3df896`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.675457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.691364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.692069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.692351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.692625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] PATH_INFO: `/v3/projects/a0041e724c9d4604a4305dd31be3ae7a/users/831e0caaf4c2407baf65dffba8561a8e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.708015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0041e724c9d4604a4305dd31be3ae7a, user_id=831e0caaf4c2407baf65dffba8561a8e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.711074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cfb87-6427-43af-b437-206e28091137 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.725581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1017/2078] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:52 2022] PUT /identity/v3/projects/a0041e724c9d4604a4305dd31be3ae7a/users/831e0caaf4c2407baf65dffba8561a8e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:52.736479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.736765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.737044 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.747193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1062/2079] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:52.758231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.759631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.772935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] MFA Rules not processed for user `831e0caaf4c2407baf65dffba8561a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:52.774160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.822477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf12cfe-2d8e-4fb3-9c6b-2be1edffe3bc None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.847979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1018/2080] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:52.863062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:52.864544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:52.882604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.883261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.883579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.883853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.887175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:52.947177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08319-c11c-44e6-b2b0-879f6b89d0f6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:52.951558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:52.952192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:52.952520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:52.952806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:52.963071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.965478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.974116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:52.976165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144051b0-72e8-48f1-a7cb-39a0c2782a90 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:52.980546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1019/2081] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:52.987955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1063/2082] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:48:53.012822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:53.096303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d79ff-adf0-4d85-a0a5-ca22fc9b0ce7 service glance] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:53.101143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:53.102327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.102605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.102874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] PATH_INFO: `/v3/users/f86173a3c86d43d9942808fb2a3df896` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.104162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86173a3c86d43d9942808fb2a3df896)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:53.105999 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:53.162319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86173a3c86d43d9942808fb2a3df896'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.162992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86173a3c86d43d9942808fb2a3df896'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.171833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86173a3c86d43d9942808fb2a3df896'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.188073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86173a3c86d43d9942808fb2a3df896'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.197163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86173a3c86d43d9942808fb2a3df896'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.209259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.209534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.209804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.211547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1020/2083] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:53 2022] DELETE /identity/v3/users/f86173a3c86d43d9942808fb2a3df896 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:53.219081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:53.235097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:53.236675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:53.315747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1064/2084] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:53.333475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:53.334826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:53.416332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:53.445127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720e55-bfac-40e3-9020-2163cbe8f454 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:53.448930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:53.449546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.449843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.450080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.467956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:53.469849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727d8a7-00d3-411b-8eb3-cddfa5807415 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:53.482833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1021/2085] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:53.776781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:53.778263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:53.791610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b68ab-df95-4d31-a38a-070368c19b21 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:53.799150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:53.799823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.800098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.800363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] PATH_INFO: `/v3/projects/f78aeb95f9714bac9460428653cfe214` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.801902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78aeb95f9714bac9460428653cfe214)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:53.803848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:53.851258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78aeb95f9714bac9460428653cfe214', 'request_id': 'req-48691fe3-2ad4-49d4-a651-67b342f9a72f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.856980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78aeb95f9714bac9460428653cfe214', 'request_id': 'req-48691fe3-2ad4-49d4-a651-67b342f9a72f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.864286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78aeb95f9714bac9460428653cfe214', 'request_id': 'req-48691fe3-2ad4-49d4-a651-67b342f9a72f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.876546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] The token cache is being invalidate because project f78aeb95f9714bac9460428653cfe214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:53.876836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.877484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-48691fe3-2ad4-49d4-a651-67b342f9a72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:53.878969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1065/2086] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:53 2022] DELETE /identity/v3/projects/f78aeb95f9714bac9460428653cfe214 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:53.884918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.885246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.885468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.904535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:53.905752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:53.965718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1022/2087] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:53.973924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:53.974201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:53.974574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:53.994886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:53.996124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:54.005916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1066/2088] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:54.011055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.021027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773ad09-92c2-42ba-9001-1f8cb0fed488 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.025519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.026126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.026404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.026695 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.027342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:54.027675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.029342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.044652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1023/2089] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:54.045743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.055644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc7402-f502-410f-9fda-8ed1a857b937 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.061082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.061690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.062419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.062419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.062856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.064788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.115928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1067/2090] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:54 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:54.118436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.130358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed265-b60e-4cd2-8436-d7516fdd5b51 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.202259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.203131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.203566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.203974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.204654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.206668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.232069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1024/2091] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:54 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:54.236257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.250278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e769f80-0726-4fd0-9c39-5c3d457988db admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.261659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.262578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.262823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.263220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.264122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.265850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.279940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1068/2092] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:48:54.281145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.292848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900b11e-27ba-41c4-a898-f73bf9b29ea9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.297727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.298332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.298610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.298883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.299758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.301486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3b178-c009-4e83-ba52-f63cfc54da01 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.312991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1025/2093] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:54.317152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.327651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1c412-44c4-4028-9203-1928002b0ca8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.332825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.333538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.333818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.334091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] PATH_INFO: `/v3/projects/c123df42588249cab1b4bc59d6fe0208/users/a0531ec1eba945b4b51d4683f2cd33db/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.354978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c123df42588249cab1b4bc59d6fe0208, user_id=a0531ec1eba945b4b51d4683f2cd33db, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.357861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.373765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1069/2094] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:54 2022] PUT /identity/v3/projects/c123df42588249cab1b4bc59d6fe0208/users/a0531ec1eba945b4b51d4683f2cd33db/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:48:54.384038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1631-73a9-471e-b290-49162361a819 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.384417 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1631-73a9-471e-b290-49162361a819 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.384763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e1631-73a9-471e-b290-49162361a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.425846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-317e1631-73a9-471e-b290-49162361a819 None None] MFA Rules not processed for user `a0531ec1eba945b4b51d4683f2cd33db`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:54.427447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-317e1631-73a9-471e-b290-49162361a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:54.472359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:54.533800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1026/2095] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:54.544960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.546668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:54.618633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:54.674441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfd25e-e2dc-4820-a787-03a2378991bd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.678094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.678702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.678979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.679250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.687604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.689032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1da20f-4cf5-4715-965f-2050be4cf6cc service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.697725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1070/2096] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:54.778967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1631-73a9-471e-b290-49162361a819 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:54.780369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-317e1631-73a9-471e-b290-49162361a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:54.865178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e1631-73a9-471e-b290-49162361a819 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:54.871397 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:54.872167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.872506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.872794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] PATH_INFO: `/v3/users/469cd3eccab4477e8e640e5a9b310872` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:54.874007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=469cd3eccab4477e8e640e5a9b310872)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:54.876169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:54.938225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469cd3eccab4477e8e640e5a9b310872'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:54.939209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469cd3eccab4477e8e640e5a9b310872'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:54.947756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469cd3eccab4477e8e640e5a9b310872'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:54.969005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469cd3eccab4477e8e640e5a9b310872'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:54.982478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469cd3eccab4477e8e640e5a9b310872'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:54.990579 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1027/2097] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:54 2022] DELETE /identity/v3/users/469cd3eccab4477e8e640e5a9b310872 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:54.995281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:54.996148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:54.996729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:55.021718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:55.023161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:55.104625 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1071/2098] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:55.115911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:55.118608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:55.191443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd64a-d359-4794-9fd3-588c8da6af2f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:55.195983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:55.196715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:55.197061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:55.197424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:55.204032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:55.208867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:55.211461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:55.224638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1028/2099] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:48:55.810925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:55.813475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:55.828229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd39e0-2cfe-4977-95c1-6b624d493f91 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:55.839724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:55.840709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:55.843422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:55.843821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] PATH_INFO: `/v3/projects/f7ca6b52c2b84d909ae0283019afe298` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:55.845313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ca6b52c2b84d909ae0283019afe298)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:55.849605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:55.915758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ca6b52c2b84d909ae0283019afe298', 'request_id': 'req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:55.926541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ca6b52c2b84d909ae0283019afe298', 'request_id': 'req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:55.939184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ca6b52c2b84d909ae0283019afe298', 'request_id': 'req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:55.952100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] The token cache is being invalidate because project f7ca6b52c2b84d909ae0283019afe298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:55.952398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:55.953048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:55.954856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1072/2100] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:55 2022] DELETE /identity/v3/projects/f7ca6b52c2b84d909ae0283019afe298 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:55.958771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:55.970544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374d868-d15e-4671-8af1-167810c868fb service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:56.063949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:56.063949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:56.063949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:56.063949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] PATH_INFO: `/v3/users/2f2167d384a94770996ae3cc78d48fe3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:56.064795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f2167d384a94770996ae3cc78d48fe3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:56.068769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:56.146849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f2167d384a94770996ae3cc78d48fe3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.150307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f2167d384a94770996ae3cc78d48fe3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.164878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f2167d384a94770996ae3cc78d48fe3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.189758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f2167d384a94770996ae3cc78d48fe3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.200713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f2167d384a94770996ae3cc78d48fe3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.208298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1029/2101] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:55 2022] DELETE /identity/v3/users/2f2167d384a94770996ae3cc78d48fe3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:56.561084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:56.563222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:56.580193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9bfb-f51d-474b-9325-984f9c0f6b9e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:56.647666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:56.672034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:56.673042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:56.673304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:56.673594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] PATH_INFO: `/v3/projects/0c72b4f38f504d1ea97a2411762be365` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:56.674715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c72b4f38f504d1ea97a2411762be365)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:56.676732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:56.735794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c72b4f38f504d1ea97a2411762be365', 'request_id': 'req-055a3d1d-20f6-4174-a0a8-86e471613727'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.741580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c72b4f38f504d1ea97a2411762be365', 'request_id': 'req-055a3d1d-20f6-4174-a0a8-86e471613727'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.749365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c72b4f38f504d1ea97a2411762be365', 'request_id': 'req-055a3d1d-20f6-4174-a0a8-86e471613727'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.760522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] The token cache is being invalidate because project 0c72b4f38f504d1ea97a2411762be365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:56.760808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.761464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.762951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1073/2102] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:56 2022] DELETE /identity/v3/projects/0c72b4f38f504d1ea97a2411762be365 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:56.767024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:56.768450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:56.781980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33455f20-36b9-45a6-bf61-46e34e1b5ad7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:56.840030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:56.840598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:56.840901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:56.841232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] PATH_INFO: `/v3/users/69a1bfdd57b14f8788b43d34e3b92edb` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:56.842354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a1bfdd57b14f8788b43d34e3b92edb)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:56.844600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:56.860249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:56.894214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a1bfdd57b14f8788b43d34e3b92edb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.894871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a1bfdd57b14f8788b43d34e3b92edb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.903111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a1bfdd57b14f8788b43d34e3b92edb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.917942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a1bfdd57b14f8788b43d34e3b92edb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.925075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3c23632-38b7-4a37-ae63-4ce06d4d3e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a1bfdd57b14f8788b43d34e3b92edb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:56.931581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1030/2103] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:48:56 2022] DELETE /identity/v3/users/69a1bfdd57b14f8788b43d34e3b92edb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:57.232894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.234712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:57.260133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3d1d-20f6-4174-a0a8-86e471613727 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.323350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.324661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.324972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.325308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] PATH_INFO: `/v3/projects/a5c9d451d3934663b634ae2aca7c1a39` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.326785 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c9d451d3934663b634ae2aca7c1a39)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.329686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.377385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c9d451d3934663b634ae2aca7c1a39', 'request_id': 'req-ae912fca-4893-4563-91a8-f04b13d2d994'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:57.383661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c9d451d3934663b634ae2aca7c1a39', 'request_id': 'req-ae912fca-4893-4563-91a8-f04b13d2d994'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:57.395301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c9d451d3934663b634ae2aca7c1a39', 'request_id': 'req-ae912fca-4893-4563-91a8-f04b13d2d994'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:57.411613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] The token cache is being invalidate because project a5c9d451d3934663b634ae2aca7c1a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:48:57.411954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:57.412628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ae912fca-4893-4563-91a8-f04b13d2d994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:48:57.414206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1074/2104] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:48:57 2022] DELETE /identity/v3/projects/a5c9d451d3934663b634ae2aca7c1a39 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:57.419135 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.419852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.419852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.438984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:57.440220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:57.499466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1031/2105] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:57.508100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897036-d325-453b-af20-51464e2e8500 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.508408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897036-d325-453b-af20-51464e2e8500 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.508501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897036-d325-453b-af20-51464e2e8500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.530300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c9897036-d325-453b-af20-51464e2e8500 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:57.531640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c9897036-d325-453b-af20-51464e2e8500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:57.540314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1075/2106] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:48:57.544835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.554664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8070227-e014-40a7-b524-c7f82f4f2d3b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.559248 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.559892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.560206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.560484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.561140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:48:57.561475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.563085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.575820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1032/2107] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:48:57.580136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897036-d325-453b-af20-51464e2e8500 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.584694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:57.591707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897036-d325-453b-af20-51464e2e8500 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.600888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.604230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.604650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.604929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.605576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.607457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.638121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1076/2108] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:48:57 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:57.642172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.652418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18bc5-e0ec-47b2-84f1-0e6c7d9a138c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.725030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.725973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.726557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.726997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.728046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.731392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.767069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1033/2109] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:48:57 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:48:57.772091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.786358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6485ad-61ab-4244-957f-046938583288 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.791329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.792041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.792379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.792712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.793422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.795097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.806952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1077/2110] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:57.811380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.821997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0409071-57ed-4089-85fc-e4cee078537b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.826788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.827390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.827732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.828021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.828690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.830317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69ddb8-3926-4d6c-9b5f-912d6cc4cc7b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.833197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:48:57.843993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1034/2111] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:48:57.848287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:57.859327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf5e41-df9f-40dc-ae83-1f75f8bbbd76 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:57.864248 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:57.864897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.865221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.865436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] PATH_INFO: `/v3/projects/473ca8b81f084f81baee856774f57eb3/users/3b8996a317c3417bbb4618d4fee1c668/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.882010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473ca8b81f084f81baee856774f57eb3, user_id=3b8996a317c3417bbb4618d4fee1c668, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:57.885451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:57.904941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1078/2112] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:48:57 2022] PUT /identity/v3/projects/473ca8b81f084f81baee856774f57eb3/users/3b8996a317c3417bbb4618d4fee1c668/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:48:57.916155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:57.916433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:57.916754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:57.943895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] MFA Rules not processed for user `3b8996a317c3417bbb4618d4fee1c668`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:48:57.945740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.014476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1035/2113] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:48:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:48:58.025533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:48:58.027312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:48:58.088059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea449b-5f3b-4d96-9430-355a97eaa361 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:48:58.092158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:48:58.092761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:48:58.093038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:48:58.093561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:48:58.102102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:48:58.103636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abddb1e-5661-47ae-97b5-f3b3d6581316 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:48:58.113700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1079/2114] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:48:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.207836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.209362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.278497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc8c66-425c-451c-b1a6-6185e17a4523 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.284986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.285693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.285974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.286244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] PATH_INFO: `/v3/users/7e69529282164956a64cb4a81fa37748` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.287371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e69529282164956a64cb4a81fa37748)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.289583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.351857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e69529282164956a64cb4a81fa37748'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:03.352598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e69529282164956a64cb4a81fa37748'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:03.360611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e69529282164956a64cb4a81fa37748'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:03.375600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e69529282164956a64cb4a81fa37748'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:03.383274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e69529282164956a64cb4a81fa37748'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:03.389577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1036/2115] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:03 2022] DELETE /identity/v3/users/7e69529282164956a64cb4a81fa37748 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:03.394119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.394616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.394879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.420440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:03.421835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.508399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1080/2116] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:03.523360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.525755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:03.536915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:03.587747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be64830-f894-413c-bb10-913c114e7131 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:03.591450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:03.592112 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:03.592395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:03.592668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:03.603863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:03.605462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:03.618191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1037/2117] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:03.984277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:03.986492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:04.008558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c6ecb-4c4e-499f-a7c4-666531a689e9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:04.020766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:04.021990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:04.022728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:04.023156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] PATH_INFO: `/v3/projects/fd70ed66c01141f5a3c193ffe503aa8f` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:04.040634 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd70ed66c01141f5a3c193ffe503aa8f)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:04.043396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:04.120209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:04.124480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd70ed66c01141f5a3c193ffe503aa8f', 'request_id': 'req-60e31012-f8c5-4ef8-9c80-8bdb38612039'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.134115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd70ed66c01141f5a3c193ffe503aa8f', 'request_id': 'req-60e31012-f8c5-4ef8-9c80-8bdb38612039'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.142354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd70ed66c01141f5a3c193ffe503aa8f', 'request_id': 'req-60e31012-f8c5-4ef8-9c80-8bdb38612039'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.154302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] The token cache is being invalidate because project fd70ed66c01141f5a3c193ffe503aa8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:04.155519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.155519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.157210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1081/2118] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:03 2022] DELETE /identity/v3/projects/fd70ed66c01141f5a3c193ffe503aa8f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:04.162316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:04.178634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32376078-3408-4a2b-895b-eb728630e2f6 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:04.266989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:04.267766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:04.268193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:04.268599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] PATH_INFO: `/v3/users/677af431cbdf4d34932024641ea905b1` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:04.269675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=677af431cbdf4d34932024641ea905b1)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:04.271681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:04.341194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677af431cbdf4d34932024641ea905b1'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.342116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677af431cbdf4d34932024641ea905b1'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.355428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677af431cbdf4d34932024641ea905b1'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.376338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677af431cbdf4d34932024641ea905b1'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.385462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-32bff0b6-893d-4d50-adb9-20bb2bd18b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677af431cbdf4d34932024641ea905b1'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:04.392581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1038/2119] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:04 2022] DELETE /identity/v3/users/677af431cbdf4d34932024641ea905b1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:04.886567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:04.888650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:04.903361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e31012-f8c5-4ef8-9c80-8bdb38612039 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:04.967173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:04.968211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:04.968623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:04.968906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] PATH_INFO: `/v3/projects/364a77db391e40769635a65da061e545` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:04.970371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364a77db391e40769635a65da061e545)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:04.972566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.024496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364a77db391e40769635a65da061e545', 'request_id': 'req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.031720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364a77db391e40769635a65da061e545', 'request_id': 'req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.050380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364a77db391e40769635a65da061e545', 'request_id': 'req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.074072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] The token cache is being invalidate because project 364a77db391e40769635a65da061e545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:05.074072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.075070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.077545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1082/2120] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:04 2022] DELETE /identity/v3/projects/364a77db391e40769635a65da061e545 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:05.084110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6d6a-d582-4928-b8d2-a125460efa38 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.087906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6d6a-d582-4928-b8d2-a125460efa38 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.088331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6d6a-d582-4928-b8d2-a125460efa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.137265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fe8c6d6a-d582-4928-b8d2-a125460efa38 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:05.140363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fe8c6d6a-d582-4928-b8d2-a125460efa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.190349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.192532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.203966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:05.320230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1039/2121] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:05.325310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.325917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.326361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.352444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:05.354371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.368071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1040/2122] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:05.376433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.387398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d6fa3-6fc4-4de3-9a48-b5d4913c97d8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.393367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef4b8-4301-4101-b05a-571b634e47d9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.398795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.399969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.400641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.401614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] PATH_INFO: `/v3/users/a0531ec1eba945b4b51d4683f2cd33db` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.403642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0531ec1eba945b4b51d4683f2cd33db)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.406571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.407508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.412385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.413130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.413682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.414463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:05.415030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:05.415400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.417471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.434065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1041/2123] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:05.440556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.458374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e9eac-be72-46e0-b12c-ded871b7463b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.464460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.465404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.465595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.465904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.466536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.468982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.483392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0531ec1eba945b4b51d4683f2cd33db'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.484549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0531ec1eba945b4b51d4683f2cd33db'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.500447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0531ec1eba945b4b51d4683f2cd33db'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.508369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1042/2124] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:05.513678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0531ec1eba945b4b51d4683f2cd33db'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.514570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.517284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.546434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4cabb471-6fca-46da-80dc-7088efbf2ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0531ec1eba945b4b51d4683f2cd33db'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:05.554795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1083/2125] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:05 2022] DELETE /identity/v3/users/a0531ec1eba945b4b51d4683f2cd33db => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:05.559834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.560225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.560663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.580746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:05.581997 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.601811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbe5d0-a5fa-407d-abca-d5c58e063aaf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.659399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1084/2126] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:05.671471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.673206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:05.673726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.674347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.674614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.674888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.676820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.678764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.710478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1043/2127] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:05.715869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.733375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2c2f-f19d-4c43-a88d-83ba50bce1dd admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.738873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.740166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.740166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.740166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.740670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.742870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.753730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1044/2128] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:05.756932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b051c9-ecf9-4dd5-b00e-c545f3d080e0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.757558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.761312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.762062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.762445 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.762950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.768701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05fdf8-9323-4695-ba9a-2ab4fab35c00 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.774986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.780259 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.781001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.781614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.781892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.782161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.782815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.784466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cb839-8068-4adf-a688-7ff85f9c43ca admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.801118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1085/2129] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:05.801346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1045/2130] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:05.805803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:05.816524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35b3c3-203a-46d7-8f77-5d8f0f2ecb09 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:05.822575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:05.823278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.823745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.824046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] PATH_INFO: `/v3/projects/9bddcbf1ca6a4bbd94fe52c2d9b91c09/users/a798ffc6ee56406ca724a176ce46d22c/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.842212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bddcbf1ca6a4bbd94fe52c2d9b91c09, user_id=a798ffc6ee56406ca724a176ce46d22c, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:05.845240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:05.856383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:05.863431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1086/2131] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:05 2022] PUT /identity/v3/projects/9bddcbf1ca6a4bbd94fe52c2d9b91c09/users/a798ffc6ee56406ca724a176ce46d22c/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:05.875423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:05.875749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:05.876144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:05.887771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:05.914336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] MFA Rules not processed for user `a798ffc6ee56406ca724a176ce46d22c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:05.916207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.017841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1046/2132] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:06.023604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.100280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb2ef2-7203-4295-8767-c16e3d4fcd00 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.109267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.109926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.111536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.112080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.113402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.115944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.167523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1087/2133] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:06.171631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.184025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cff6b-55a3-4bb8-b938-cd7cd63dab19 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.255292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.255946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.256305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.256660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.257915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.260386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.281153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.283010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.294639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1047/2134] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:06.298287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.306550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58bfaf-22cd-4b7f-a9b4-af8efd0a1b81 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.322538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.323716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.324022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.324285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] PATH_INFO: `/v3/projects/c123df42588249cab1b4bc59d6fe0208` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.325540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c123df42588249cab1b4bc59d6fe0208)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.328309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.334422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f889-288d-42d7-948b-3cbcfc227c70 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.348270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.349822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.350113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.350398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.352367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.355792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.386791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1048/2135] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:06.389439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.414382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29493be4-b1b2-4f30-80a0-4852d709088a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.416595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c123df42588249cab1b4bc59d6fe0208', 'request_id': 'req-5562b32f-bd1a-48bb-8253-e2466b23c9b8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:06.428060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c123df42588249cab1b4bc59d6fe0208', 'request_id': 'req-5562b32f-bd1a-48bb-8253-e2466b23c9b8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:06.450429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c123df42588249cab1b4bc59d6fe0208', 'request_id': 'req-5562b32f-bd1a-48bb-8253-e2466b23c9b8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:06.537963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] The token cache is being invalidate because project c123df42588249cab1b4bc59d6fe0208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:06.538936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:06.540019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5562b32f-bd1a-48bb-8253-e2466b23c9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:06.545180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1088/2136] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:06 2022] DELETE /identity/v3/projects/c123df42588249cab1b4bc59d6fe0208 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:06.552192 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5962f-a5a0-4a13-a34e-3067e370470d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.552668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5962f-a5a0-4a13-a34e-3067e370470d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.553126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5962f-a5a0-4a13-a34e-3067e370470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.587489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-20c5962f-a5a0-4a13-a34e-3067e370470d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:06.588247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.589541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-20c5962f-a5a0-4a13-a34e-3067e370470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.590230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.591306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.592581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] PATH_INFO: `/v3/projects/5dd2be3c97074eb7af12f17efa620545/users/845631d20c0a4eadbb2a4682f01ea426/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.604158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1089/2137] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:06.616246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.617296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.617922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.637256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd2be3c97074eb7af12f17efa620545, user_id=845631d20c0a4eadbb2a4682f01ea426, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.640749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.645159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:06.666754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:06.670208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.682069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1049/2138] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:06 2022] PUT /identity/v3/projects/5dd2be3c97074eb7af12f17efa620545/users/845631d20c0a4eadbb2a4682f01ea426/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:06.686222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.688639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:06.786672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1090/2139] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:06.788121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.796262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:06.804152 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602e8a-92b9-4394-8304-1d36900b4cdd None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.810772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.812124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.812976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.813352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.814069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:06.814589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.817038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.832550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1091/2140] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:49:06.839783 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.848267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f814b0b-5c55-455a-8c59-abf19124a53a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.867665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.868734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.869237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.869729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.870929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.881373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.889076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5118e-2d2b-4722-aaa3-a66bf3d1d908 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.911280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1050/2141] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:06.918064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:06.947208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318e897-f5d0-49e3-8d50-3d32a18c49dc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:06.952307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.953144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.953605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.954014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.954711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.957926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:06.958408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.959061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:06.959589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:06.960720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] PATH_INFO: `/v3/projects/5dd2be3c97074eb7af12f17efa620545/users/845631d20c0a4eadbb2a4682f01ea426/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:06.963155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd2be3c97074eb7af12f17efa620545, user_id=845631d20c0a4eadbb2a4682f01ea426, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:06.966786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e5c90-0d57-4800-90b4-9abb2363fea2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:06.985648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1051/2142] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:06 2022] PUT /identity/v3/projects/5dd2be3c97074eb7af12f17efa620545/users/845631d20c0a4eadbb2a4682f01ea426/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:07.004274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.004558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.004840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.010683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1092/2143] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:07.014984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.058699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] MFA Rules not processed for user `845631d20c0a4eadbb2a4682f01ea426`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:07.061089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.114384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a2b5-e65e-4559-a087-15ad2e5ecca7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.119454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.120167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.120517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.120864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.121667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.124869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.149852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1052/2144] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:07.153528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.154912 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1093/2145] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:07 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:07.162221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.163854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.166960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd6468-9626-4a86-87f6-fa7e37c8fd54 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.172309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.173309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.173751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.174186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.174885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.176469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.191527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1053/2146] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:07.191527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.207110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd220f-299d-4be3-9bd9-bfd1a294966c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.212785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.213535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.213823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.214098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.214745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.216394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.227375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1054/2147] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:07.231850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a035-1e05-4682-8b72-eac226f68f90 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.231989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.235914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.236621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.237000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.237532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.242609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830248-6ba5-410e-8ec4-3b9c46ae1050 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.247583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.248033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.248700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.248982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.249259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] PATH_INFO: `/v3/projects/f62f652ab66a4289be10048e1c8169d6/users/4aa761298e1f45e08fe4e42fba210d2b/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.249368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.271247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1094/2148] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:07.283770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62f652ab66a4289be10048e1c8169d6, user_id=4aa761298e1f45e08fe4e42fba210d2b, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.287472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a32be4-890d-4895-ac3b-83db7f731750 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.300353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.303161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1055/2149] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:07 2022] PUT /identity/v3/projects/f62f652ab66a4289be10048e1c8169d6/users/4aa761298e1f45e08fe4e42fba210d2b/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:07.308122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.308425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.308727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.316658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea60b5-2257-48fb-ae35-93faff4c2a2a service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.331752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:07.332962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.345379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:07.417975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1056/2150] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:07.423261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.423982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.424407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.424687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] PATH_INFO: `/v3/users/a798ffc6ee56406ca724a176ce46d22c` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.425847 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a798ffc6ee56406ca724a176ce46d22c)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.428072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.428220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.430166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.430361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:07.500891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a798ffc6ee56406ca724a176ce46d22c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:07.504218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a798ffc6ee56406ca724a176ce46d22c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:07.514800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9f293-cf20-46c6-9ea1-2f11b361a8ad None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.528558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.528746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a798ffc6ee56406ca724a176ce46d22c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:07.528838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.528838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.529002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.531530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.538464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a798ffc6ee56406ca724a176ce46d22c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:07.548053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.550030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.552111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9edd1e2f-1557-451d-a5c8-d50787cf8772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a798ffc6ee56406ca724a176ce46d22c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:07.558787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1095/2151] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:07 2022] DELETE /identity/v3/users/a798ffc6ee56406ca724a176ce46d22c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:07.560972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1057/2152] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:07.563024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.563315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.563638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.583053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:07.584332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.655375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1096/2153] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:07.666596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:07.668355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:07.680553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620c9d0-6176-409d-8270-7a5e58bfe979 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:07.734516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:07.735143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:07.735448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:07.735762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:07.757232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:07.760843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c945ec-89dd-4f99-92e8-dd060ac7cc71 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:07.772400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1058/2154] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:08.401488 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:08.404522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.418596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12c872-aa97-4d16-aae7-c98e090527a3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:08.432664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:08.433383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.433768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.434101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] PATH_INFO: `/v3/projects/9bddcbf1ca6a4bbd94fe52c2d9b91c09` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.446678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bddcbf1ca6a4bbd94fe52c2d9b91c09)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:08.450517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:08.507934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.508489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.508489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.539166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bddcbf1ca6a4bbd94fe52c2d9b91c09', 'request_id': 'req-6082db74-f7d8-408b-9f45-aff217ce5414'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.541886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] MFA Rules not processed for user `4aa761298e1f45e08fe4e42fba210d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:08.543416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.545784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bddcbf1ca6a4bbd94fe52c2d9b91c09', 'request_id': 'req-6082db74-f7d8-408b-9f45-aff217ce5414'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.557372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bddcbf1ca6a4bbd94fe52c2d9b91c09', 'request_id': 'req-6082db74-f7d8-408b-9f45-aff217ce5414'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.573353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] The token cache is being invalidate because project 9bddcbf1ca6a4bbd94fe52c2d9b91c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:08.573757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.575348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.577245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1097/2155] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:08 2022] DELETE /identity/v3/projects/9bddcbf1ca6a4bbd94fe52c2d9b91c09 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:08.581173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:08.583459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.590837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:08.636400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:08.659065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1059/2156] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:08.670448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db74-f7d8-408b-9f45-aff217ce5414 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:08.671222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:08.672750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.675134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:08.675800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.676074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.676340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] PATH_INFO: `/v3/users/845631d20c0a4eadbb2a4682f01ea426` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.677456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=845631d20c0a4eadbb2a4682f01ea426)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:08.685456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:08.739194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845631d20c0a4eadbb2a4682f01ea426'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.740025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845631d20c0a4eadbb2a4682f01ea426'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.748610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845631d20c0a4eadbb2a4682f01ea426'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.755830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845631d20c0a4eadbb2a4682f01ea426'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.761681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936541e-053c-4634-8f8e-2edc6217ae73 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:08.765409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:08.766016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:08.766293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:08.766564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:08.768406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:08.769309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845631d20c0a4eadbb2a4682f01ea426'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:08.775808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1098/2157] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:08 2022] DELETE /identity/v3/users/845631d20c0a4eadbb2a4682f01ea426 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:08.779834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:08.781284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc5689-b6f0-42b6-b0a9-62acabdfc93f service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:08.791052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1060/2158] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:09.128193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.130467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:09.224878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d694f-de71-4a59-bc30-914c32e8fbad admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.230697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.231450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.231761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.232034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] PATH_INFO: `/v3/projects/5dd2be3c97074eb7af12f17efa620545` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.233155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd2be3c97074eb7af12f17efa620545)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.235102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.260642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:09.283998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd2be3c97074eb7af12f17efa620545', 'request_id': 'req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:09.289511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd2be3c97074eb7af12f17efa620545', 'request_id': 'req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:09.297890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd2be3c97074eb7af12f17efa620545', 'request_id': 'req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:09.308727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] The token cache is being invalidate because project 5dd2be3c97074eb7af12f17efa620545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:09.309014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:09.309639 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e217ed7-d69a-4894-8116-5dcf1b9c1fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:09.311135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1099/2159] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:09 2022] DELETE /identity/v3/projects/5dd2be3c97074eb7af12f17efa620545 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:09.316854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.317145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.317424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.339410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:09.340668 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:09.400545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1061/2160] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:09.408378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.408890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.408987 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.430337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:09.431708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:09.439610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1100/2161] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:09.443785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.453653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8522247-3ceb-4d8d-9ddb-d582b9a6b912 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.458282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.458881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.459161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.459431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.460131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:09.460435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.462056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.476494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1062/2162] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:09.478534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.487707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb90f51-491c-4c45-a814-92ae13d4310f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.493269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.493921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.494162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.494386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.495005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.496925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.527661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1101/2163] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:09.531965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.543041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23910b5-5073-427b-a568-0f1c94813b52 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.600213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.600854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.601136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.601409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.602031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.604046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.628010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1063/2164] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:09.632732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.644209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c0916-ba95-4cb0-8fe1-d959e4fd0dfa admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.649841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.650572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.650855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.651128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.651805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.653410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.670556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1102/2165] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:09.674658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.684620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde243c-8a2b-47e4-afd6-cb6d087d8c6a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.689321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.689915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.690190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.690458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.691093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.692733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338212d4-6bda-49c2-a6ab-2e8c4bb0ff3d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.697956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:09.704547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1064/2166] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:09.707866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.718913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacdaad-3843-4714-853c-4dba1f4d08cd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.723620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.724269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.724555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.724828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] PATH_INFO: `/v3/projects/5abc14e9816c4c28addba743a0fec82f/users/24225c75b8d945a7b5c0bcd0173162c2/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.739538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5abc14e9816c4c28addba743a0fec82f, user_id=24225c75b8d945a7b5c0bcd0173162c2, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.742254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.754711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1103/2167] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:09 2022] PUT /identity/v3/projects/5abc14e9816c4c28addba743a0fec82f/users/24225c75b8d945a7b5c0bcd0173162c2/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:09.766208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.766426 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.766696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.795187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] MFA Rules not processed for user `24225c75b8d945a7b5c0bcd0173162c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:09.796561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:09.860565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1065/2168] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:09.865112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.924827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556ec2d-2085-4a41-a3f3-43529a038442 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:09.929447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:09.930034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:09.930311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:09.930574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:09.931179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:09.933100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:09.958894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1104/2169] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:09.963414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:09.974379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7d716-a756-4149-9d4c-5fe5d30049dc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.030104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.030786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.031091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.031425 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.032180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.034323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.055905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1066/2170] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:09 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:10.060064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.070877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdfd14-4ac0-48be-a02b-be8430e74d30 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.075575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.076167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.076449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.076725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.077382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.078964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.090353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1105/2171] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:10.094660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.104527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71502f-8614-43a3-abaf-8aaf4409ba2c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.109162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.109745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.110016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.110283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.110911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.112534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced70b76-a4d0-436b-8556-291721c1a196 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.122116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1067/2172] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:10.126051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.137008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2763ee-3ee0-4dbb-9386-4221153fa4ef admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.141752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.142395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.142678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.142950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] PATH_INFO: `/v3/projects/8cc1b4e1c31a44bab00dd662dec8fde3/users/5a98a94af0054a6ab70875e818baa3aa/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.158027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc1b4e1c31a44bab00dd662dec8fde3, user_id=5a98a94af0054a6ab70875e818baa3aa, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.160798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.169474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:10.178013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1106/2173] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:10 2022] PUT /identity/v3/projects/8cc1b4e1c31a44bab00dd662dec8fde3/users/5a98a94af0054a6ab70875e818baa3aa/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:10.189071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.189388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.190065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.221798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] MFA Rules not processed for user `5a98a94af0054a6ab70875e818baa3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:10.223025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:10.284011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:10.295525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1068/2174] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:10.296968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.359935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008881be-4745-40b9-bf90-17339e3cd000 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.364663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.365266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.365544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.365821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.367012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.368956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.397241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1107/2175] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:10 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:10.402410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.420513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17857bf6-4114-427d-8b87-3df471523cb1 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.482797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.483410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.483718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.483993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.484612 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.486538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.506936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1069/2176] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:10 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:10.510960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.521348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106a8fe-a25d-4bf2-bf65-1ce9b88a1fa7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.526853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.527516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.527809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.528085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.528924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.530620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.542675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1108/2177] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:10.546925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.557311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068675e-2a7d-411e-8bba-a90da7cfebff admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.562144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.562828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.563111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.563387 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] PATH_INFO: `/v3/projects/4f127dff9a84443dbfd1981eb093b929/users/b704a29b72134108832dcb19131c7291/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.581937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f127dff9a84443dbfd1981eb093b929, user_id=b704a29b72134108832dcb19131c7291, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.584798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.597970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1070/2178] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:10 2022] PUT /identity/v3/projects/4f127dff9a84443dbfd1981eb093b929/users/b704a29b72134108832dcb19131c7291/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:10.602438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.612595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08eaf6-8292-4f30-b45f-ed56a4fe42d6 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.670447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.671062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.671342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.671653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.672295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.673898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3468-6b51-4ad6-ae68-61fea52994e3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.683822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1109/2179] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:10.688284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.706944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00d898-2b0a-4b0e-bf41-1b073443b500 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.712239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.712904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.713184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.713454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] PATH_INFO: `/v3/projects/4f127dff9a84443dbfd1981eb093b929/users/b704a29b72134108832dcb19131c7291/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.715337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f127dff9a84443dbfd1981eb093b929, user_id=b704a29b72134108832dcb19131c7291, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.718244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.730263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1071/2180] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:10 2022] PUT /identity/v3/projects/4f127dff9a84443dbfd1981eb093b929/users/b704a29b72134108832dcb19131c7291/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:10.740583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.740874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.741181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.768323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] MFA Rules not processed for user `b704a29b72134108832dcb19131c7291`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:10.769547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:10.826367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:10.838442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1110/2181] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:10.849767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.851564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:10.909961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc31-dbe9-4adf-9552-0e7f0b336615 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.913686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.914289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.914570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.914846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:10.923760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:10.925193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ba737-0424-41a6-a3e8-70f420a80189 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:10.934788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1072/2182] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:10.975745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:10.977241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:10.989246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c040f7c-a27f-4918-8104-88e0ef71ad9f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:10.992865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:10.993457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:10.993733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:10.994002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:11.049769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:11.051198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:11.060040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1111/2183] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:13.396511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff0ea-3d68-4f5a-b5d5-7dbb5811172d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:13.397042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff0ea-3d68-4f5a-b5d5-7dbb5811172d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:13.397173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedff0ea-3d68-4f5a-b5d5-7dbb5811172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:13.416676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-bedff0ea-3d68-4f5a-b5d5-7dbb5811172d None None] MFA Rules not processed for user `4aa761298e1f45e08fe4e42fba210d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:13.417880 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bedff0ea-3d68-4f5a-b5d5-7dbb5811172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:13.465280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:13.481879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1073/2184] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.232150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:15.300718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f5708-96c2-487e-8c5a-4322b78daa20 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:15.305852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:15.306506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.306782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.307069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] PATH_INFO: `/v3/users/24225c75b8d945a7b5c0bcd0173162c2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.308193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=24225c75b8d945a7b5c0bcd0173162c2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:15.310125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:15.358520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24225c75b8d945a7b5c0bcd0173162c2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.359436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24225c75b8d945a7b5c0bcd0173162c2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.368002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24225c75b8d945a7b5c0bcd0173162c2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.379739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24225c75b8d945a7b5c0bcd0173162c2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.395715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24225c75b8d945a7b5c0bcd0173162c2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:15.401794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1112/2185] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:15 2022] DELETE /identity/v3/users/24225c75b8d945a7b5c0bcd0173162c2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:15.406211 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.406678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.406792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.426048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:15.427264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.510974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1074/2186] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:15.521827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:15.523668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.544408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:15.599757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41bbb-badb-422c-9afe-dd15dcef02ac admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:15.604311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:15.604926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.605203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.605473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.616169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:15.617635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:15.628698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1113/2187] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:15.917180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:15.919050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:15.933103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798cd78-e813-43ef-a323-ef8281c26824 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:15.938598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:15.939512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:15.939941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:15.940406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] PATH_INFO: `/v3/projects/5abc14e9816c4c28addba743a0fec82f` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:15.941717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5abc14e9816c4c28addba743a0fec82f)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:15.944247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:15.997202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5abc14e9816c4c28addba743a0fec82f', 'request_id': 'req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.002965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5abc14e9816c4c28addba743a0fec82f', 'request_id': 'req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.011108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5abc14e9816c4c28addba743a0fec82f', 'request_id': 'req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.022202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] The token cache is being invalidate because project 5abc14e9816c4c28addba743a0fec82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:16.022486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.023718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.025200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1075/2188] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:15 2022] DELETE /identity/v3/projects/5abc14e9816c4c28addba743a0fec82f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:16.029147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.040573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7c4e9-c07d-419d-98be-b85178b98e8b service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.098632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.099270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.099576 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.099851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] PATH_INFO: `/v3/users/5a98a94af0054a6ab70875e818baa3aa` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.100984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a98a94af0054a6ab70875e818baa3aa)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.103031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.152350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a98a94af0054a6ab70875e818baa3aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.153554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a98a94af0054a6ab70875e818baa3aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.161514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a98a94af0054a6ab70875e818baa3aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.168537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a98a94af0054a6ab70875e818baa3aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.180269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a98a94af0054a6ab70875e818baa3aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.186050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1114/2189] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:16 2022] DELETE /identity/v3/users/5a98a94af0054a6ab70875e818baa3aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:16.523114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.524692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.546696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd375cf-e9d4-453c-8bf7-4c26e4345b46 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.607065 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.607826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.608105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.608371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] PATH_INFO: `/v3/projects/8cc1b4e1c31a44bab00dd662dec8fde3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.609795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc1b4e1c31a44bab00dd662dec8fde3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.611938 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.640666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:16.666345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc1b4e1c31a44bab00dd662dec8fde3', 'request_id': 'req-2e23d330-0d3c-4153-a150-7ac3f6ad8140'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.673384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc1b4e1c31a44bab00dd662dec8fde3', 'request_id': 'req-2e23d330-0d3c-4153-a150-7ac3f6ad8140'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.687092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc1b4e1c31a44bab00dd662dec8fde3', 'request_id': 'req-2e23d330-0d3c-4153-a150-7ac3f6ad8140'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.702551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] The token cache is being invalidate because project 8cc1b4e1c31a44bab00dd662dec8fde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:16.703823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.706263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.712140 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1076/2190] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:16 2022] DELETE /identity/v3/projects/8cc1b4e1c31a44bab00dd662dec8fde3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:16.713076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:16.714535 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:16.729279 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a170be8-67f7-47ba-86e1-ea19f90b4504 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:16.766353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:16.798704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:16.799156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:16.799589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:16.799869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] PATH_INFO: `/v3/users/b704a29b72134108832dcb19131c7291` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:16.801072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b704a29b72134108832dcb19131c7291)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:16.808132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:16.881296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b704a29b72134108832dcb19131c7291'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.881948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b704a29b72134108832dcb19131c7291'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.890736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b704a29b72134108832dcb19131c7291'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.899168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b704a29b72134108832dcb19131c7291'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.915038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-45d602dd-226c-46c0-a6f9-5a395542954a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b704a29b72134108832dcb19131c7291'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:16.920960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1115/2191] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:16 2022] DELETE /identity/v3/users/b704a29b72134108832dcb19131c7291 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:17.195833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.197343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:17.221642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d330-0d3c-4153-a150-7ac3f6ad8140 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.280977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.281667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.282035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.282220 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] PATH_INFO: `/v3/projects/4f127dff9a84443dbfd1981eb093b929` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.283464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f127dff9a84443dbfd1981eb093b929)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.285471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.355858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f127dff9a84443dbfd1981eb093b929', 'request_id': 'req-d29b3a1e-a506-4ba4-90cb-5171e276e722'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.363054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f127dff9a84443dbfd1981eb093b929', 'request_id': 'req-d29b3a1e-a506-4ba4-90cb-5171e276e722'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.374415 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f127dff9a84443dbfd1981eb093b929', 'request_id': 'req-d29b3a1e-a506-4ba4-90cb-5171e276e722'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.394836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] The token cache is being invalidate because project 4f127dff9a84443dbfd1981eb093b929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:17.395134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.395966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d29b3a1e-a506-4ba4-90cb-5171e276e722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:17.399399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1077/2192] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:17 2022] DELETE /identity/v3/projects/4f127dff9a84443dbfd1981eb093b929 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:17.405089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.405459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.405892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.452999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:17.454475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:17.576538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1116/2193] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:17.586155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.586710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.586710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.606571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:17.608080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:17.616067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1078/2194] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:17.620362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.630114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4841b4-2937-42c8-8041-8a161b55e16d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.635207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.635874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.636145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.636401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.637050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:17.637351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.638966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.655787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1117/2195] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:17.669175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.669175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18096f6d-5cd3-4c5b-8ff0-6fe09f40ac9e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.674618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.675250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.675572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.675845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.676481 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.678532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.711968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1079/2196] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:17 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:17.713045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.723932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d174d-33e9-490d-802c-ce3d681ed6f7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.763450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:17.785591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.786532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.790071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.790071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.790071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.790071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.816227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1118/2197] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:17 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:17.817611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.828908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618c18a-fee3-4c9f-9583-7ca5fcd322a9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.834138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.835150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.835894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.836303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.837049 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.838733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.853913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1080/2198] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:17.859440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.869076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd6a88-24dc-400b-8139-27138b151655 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.876142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.877013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.877499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.878005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.878782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.880816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc2764-abf8-478a-81b7-e8bdab2ea1f7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:17.911692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1119/2199] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:17.916945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:17.925006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:17.933933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7333e-7503-4b85-b144-42884bd4e292 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:17.941750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:17.942909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:17.942909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:17.943113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] PATH_INFO: `/v3/projects/728cab1693854f2ca0efeeac2d2a8a07/users/76396e0222fb4fb7adf69136aea2a705/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:17.971099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728cab1693854f2ca0efeeac2d2a8a07, user_id=76396e0222fb4fb7adf69136aea2a705, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:17.974310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.001946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1081/2200] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:17 2022] PUT /identity/v3/projects/728cab1693854f2ca0efeeac2d2a8a07/users/76396e0222fb4fb7adf69136aea2a705/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:18.013975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.013975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.014237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.059541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] MFA Rules not processed for user `76396e0222fb4fb7adf69136aea2a705`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:18.061313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.158965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1120/2201] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:18.160477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.265504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf95ce-381e-49d9-b26c-0902cb449af3 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.272855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.274194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.274644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.275061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.275881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.278174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.312379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1082/2202] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:18.313851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.326688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a4bf0a-dfcf-4ecb-85b1-f79d764a2df9 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.422124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.422742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.423024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.423298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.423967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.427213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.457903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1121/2203] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:18.459436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.482750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83566e-3580-4162-9482-6a91073691ca admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.487868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.489074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.490075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.490075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.490756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.492570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.504697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1083/2204] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:18.509767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.521018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138c137-7528-4472-ba30-a94d5bacf4bd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.526782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.527709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.528051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.528396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] PATH_INFO: `/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc/users/6497de93bccd4ff8a9f1aa0c7461d02b/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.554093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e068ec0b76ba4e6eaef3c9751f101ebc, user_id=6497de93bccd4ff8a9f1aa0c7461d02b, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.561345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.585929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1122/2205] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:18 2022] PUT /identity/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc/users/6497de93bccd4ff8a9f1aa0c7461d02b/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:18.590024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.606043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cd12-9c6f-4178-a00e-dbfb4b77a246 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.693863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.695391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.695787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.696363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.697212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.699206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc76a3-915b-425e-bb7b-bcd404ee8160 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.711842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1084/2206] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:18.715892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.730617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8990e0-540c-4be6-b07d-a5b5905b5939 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:18.737325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:18.738140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.738537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.738908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] PATH_INFO: `/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc/users/6497de93bccd4ff8a9f1aa0c7461d02b/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.742306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e068ec0b76ba4e6eaef3c9751f101ebc, user_id=6497de93bccd4ff8a9f1aa0c7461d02b, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:18.745375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:18.754070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:18.767733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1123/2207] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:18 2022] PUT /identity/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc/users/6497de93bccd4ff8a9f1aa0c7461d02b/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:18.770369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:18.770945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:18.771311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:18.830785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] MFA Rules not processed for user `6497de93bccd4ff8a9f1aa0c7461d02b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:18.832333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:18.900667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:18.939606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1085/2208] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:18.951887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:18.954265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:19.075848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf803-adb9-42dd-926f-dd1193f8dbc7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.079806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.080430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.080711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.080982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.091209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.093002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77099d51-fb75-47d8-846c-8f813f115b38 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.106724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1124/2209] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:19.825323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:19.905263 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8b44-e919-4bd2-ac19-bd2c5a90cc93 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:19.910895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:19.911574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:19.911857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:19.912141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] PATH_INFO: `/v3/users/76396e0222fb4fb7adf69136aea2a705` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:19.913306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76396e0222fb4fb7adf69136aea2a705)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:19.916272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:19.979436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76396e0222fb4fb7adf69136aea2a705'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.980196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76396e0222fb4fb7adf69136aea2a705'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:19.987940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76396e0222fb4fb7adf69136aea2a705'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.004247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76396e0222fb4fb7adf69136aea2a705'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.023949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76396e0222fb4fb7adf69136aea2a705'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.033267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1086/2210] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:19 2022] DELETE /identity/v3/users/76396e0222fb4fb7adf69136aea2a705 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:20.038165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.039228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.039773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.082001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:20.083450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:20.175234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1125/2211] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:20.185434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.187343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:20.253357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cfdcd-b95f-451c-b51a-66f7fbac47e0 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.256957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.258118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.258391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.258751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.265523 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:20.268086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.270429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.281296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1087/2212] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:20.676642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.678243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:20.712438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348fb47-8dc6-4861-8552-9eaa655bfad6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.717116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.718177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.718616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.719043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] PATH_INFO: `/v3/projects/728cab1693854f2ca0efeeac2d2a8a07` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.720360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=728cab1693854f2ca0efeeac2d2a8a07)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.722274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.775432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728cab1693854f2ca0efeeac2d2a8a07', 'request_id': 'req-78f8d241-20fa-4281-9a90-9b1e88c620aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.781004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728cab1693854f2ca0efeeac2d2a8a07', 'request_id': 'req-78f8d241-20fa-4281-9a90-9b1e88c620aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.789087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728cab1693854f2ca0efeeac2d2a8a07', 'request_id': 'req-78f8d241-20fa-4281-9a90-9b1e88c620aa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.800633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] The token cache is being invalidate because project 728cab1693854f2ca0efeeac2d2a8a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:20.800920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.801569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.803328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1126/2213] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:20 2022] DELETE /identity/v3/projects/728cab1693854f2ca0efeeac2d2a8a07 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:20.808256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:20.817744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e8b15-4001-4485-b0f5-e10b57a03e27 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:20.874975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:20.875710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:20.875908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:20.876177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] PATH_INFO: `/v3/users/6497de93bccd4ff8a9f1aa0c7461d02b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:20.877409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6497de93bccd4ff8a9f1aa0c7461d02b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:20.879400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:20.928755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6497de93bccd4ff8a9f1aa0c7461d02b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.929413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6497de93bccd4ff8a9f1aa0c7461d02b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.937277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6497de93bccd4ff8a9f1aa0c7461d02b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.949444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6497de93bccd4ff8a9f1aa0c7461d02b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.956192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d9b8ea4f-6140-4dbb-8e95-3d556a05dea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6497de93bccd4ff8a9f1aa0c7461d02b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:20.962658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1088/2214] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:20 2022] DELETE /identity/v3/users/6497de93bccd4ff8a9f1aa0c7461d02b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:21.321532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.328005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.343674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8d241-20fa-4281-9a90-9b1e88c620aa admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.372213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:21.408075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.408712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.408990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.409263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] PATH_INFO: `/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.410387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e068ec0b76ba4e6eaef3c9751f101ebc)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.412940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.465069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e068ec0b76ba4e6eaef3c9751f101ebc', 'request_id': 'req-24632385-2ad5-4a32-a4de-627ce2fd6065'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.470893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e068ec0b76ba4e6eaef3c9751f101ebc', 'request_id': 'req-24632385-2ad5-4a32-a4de-627ce2fd6065'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.478950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e068ec0b76ba4e6eaef3c9751f101ebc', 'request_id': 'req-24632385-2ad5-4a32-a4de-627ce2fd6065'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.489950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] The token cache is being invalidate because project e068ec0b76ba4e6eaef3c9751f101ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:21.490236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.490859 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24632385-2ad5-4a32-a4de-627ce2fd6065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:21.492932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1127/2215] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:21 2022] DELETE /identity/v3/projects/e068ec0b76ba4e6eaef3c9751f101ebc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:21.497820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.498006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.498376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.517657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:21.518863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.592632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1089/2216] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:21.598027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.598338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.598759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.621588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:21.622836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:21.631806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1128/2217] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:21.636921 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.646480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1c538-2a55-4be5-862a-492e3788b58e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.651228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.651866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.652142 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.652409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.653046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:21.653345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.654941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.666482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1090/2218] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:21.670917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.680689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7826c0b-19ad-418f-ac10-24dd1e1eec21 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.685437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.686035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.686304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.686573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.687191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.689156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.718410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1129/2219] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:21.722382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.727590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:21.733840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab4e6a-425e-44ba-ae27-59614adee300 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.795591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.796203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.796479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.796875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.797509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.811097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.821907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1091/2220] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:21.826470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.836761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45121581-ada1-42fb-aaf2-b424707b7264 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.841312 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.841897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.842170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.842702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.843325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.844950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.854866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1130/2221] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:21.859090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.869936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c70f74-dcbb-449b-8a3e-2b8c43688270 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.874642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.875234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.875812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.875812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.876421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.878015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebcfbc-3da8-40ca-8ad9-372a2a4cee2a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.889002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1092/2222] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:21.892972 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:21.903703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf47592-704f-4c79-8463-8243ea8d456c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:21.908977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:21.909667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.909942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.910234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] PATH_INFO: `/v3/projects/e164e46e8f3643babea969dd5e4d08f5/users/9438dde54dda4baab42a5fc448dfb7f8/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.925804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e164e46e8f3643babea969dd5e4d08f5, user_id=9438dde54dda4baab42a5fc448dfb7f8, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:21.928516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:21.941928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1131/2223] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:21 2022] PUT /identity/v3/projects/e164e46e8f3643babea969dd5e4d08f5/users/9438dde54dda4baab42a5fc448dfb7f8/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:21.947630 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1434a2-5f4b-4472-86dc-e900929185b2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:21.947914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1434a2-5f4b-4472-86dc-e900929185b2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:21.948218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1434a2-5f4b-4472-86dc-e900929185b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:21.968786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fc1434a2-5f4b-4472-86dc-e900929185b2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:21.970160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fc1434a2-5f4b-4472-86dc-e900929185b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.033772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1093/2224] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:22.044257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:22.045959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.104908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe699-1927-4caf-a471-056e55e5f8bb admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:22.108658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:22.109256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.109543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.109811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.117264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:22.119732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:22.122069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:22.132941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1132/2225] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:22.885164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:22.885669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:22.885817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:22.914484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] MFA Rules not processed for user `9438dde54dda4baab42a5fc448dfb7f8`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:22.915829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:22.989565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1094/2226] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:22.994078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.003618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5775a-1685-41be-8353-dea77998e28a service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.008701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.009295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.009631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.009911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.010529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.012746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.048246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1133/2227] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:22 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:23.049862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.061129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59be64-454a-46ea-ad75-7daddf29b1b8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.070048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:23.124186 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.124799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.125080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.125350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.125970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.127935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.149623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1095/2228] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:23 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:23.153964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.164999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d520a-adbd-4217-9edd-c2240d093ae9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.169851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.170686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.171113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.171401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.172067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.173654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.185563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1134/2229] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:23.190234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.200351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69de2-fd3a-4131-9d3b-34ce07996820 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.205008 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.205656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.205934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.206207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] PATH_INFO: `/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84/users/61117f822d0a4e39a6443e77741898b1/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.221502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc19f008f1ea4723913ee7de9d3a7b84, user_id=61117f822d0a4e39a6443e77741898b1, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.224245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.238755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1096/2230] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:23 2022] PUT /identity/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84/users/61117f822d0a4e39a6443e77741898b1/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:23.242767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.252871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de108f-8179-4069-9bc3-9a079beac5b7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.309105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.309725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.310005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.310273 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.310920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.312659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363932-50ad-40a2-a155-c01037bb5531 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.322803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1135/2231] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:23.327176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:23.338445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cac76-78cf-4c74-a929-c68fc99c3d24 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:23.343367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:23.344058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:23.344338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:23.344610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] PATH_INFO: `/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84/users/61117f822d0a4e39a6443e77741898b1/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:23.346554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc19f008f1ea4723913ee7de9d3a7b84, user_id=61117f822d0a4e39a6443e77741898b1, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:23.349237 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:23.360490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1097/2232] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:23 2022] PUT /identity/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84/users/61117f822d0a4e39a6443e77741898b1/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:24.263703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.264272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.264272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.292215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] MFA Rules not processed for user `61117f822d0a4e39a6443e77741898b1`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:24.293846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:24.364559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1136/2233] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:24.376373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:24.378216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:24.437329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77e6a5-0580-495e-9c6f-05adf96bdf97 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:24.442102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:24.442712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:24.442993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:24.443265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:24.499762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:24.501216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:24.511197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1098/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:28.078654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:28.091364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ea99c-9aa5-4c01-bc2e-2a131ed9f073 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:28.095460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:28.096305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:28.096646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:28.096979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:28.107472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:28.108949 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcfde84-c7bc-44fb-a944-e9c9fcec09e1 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:28.120179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1137/2235] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:29.170035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.171581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.198821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:29.258235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058eac0-2760-4a41-a6af-a2be2aa73909 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.264546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.265452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.265821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.266213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] PATH_INFO: `/v3/users/831e0caaf4c2407baf65dffba8561a8e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.267594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=831e0caaf4c2407baf65dffba8561a8e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.269851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.341403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831e0caaf4c2407baf65dffba8561a8e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:29.342494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831e0caaf4c2407baf65dffba8561a8e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:29.350572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831e0caaf4c2407baf65dffba8561a8e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:29.368663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831e0caaf4c2407baf65dffba8561a8e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:29.378899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831e0caaf4c2407baf65dffba8561a8e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:29.387821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1099/2236] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:29 2022] DELETE /identity/v3/users/831e0caaf4c2407baf65dffba8561a8e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:29.392243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.393492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.393995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.446931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:29.448662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.480737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:29.545808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1138/2237] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:29.558318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:29.559853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:29.631233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5950aa-48e1-450c-9dca-cd497a164a5a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:29.635933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:29.636353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:29.636633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:29.636899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:29.651441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:29.654859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e636a67-46e8-4be8-99c1-5a3bf902c876 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:29.669416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1100/2238] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:30.252425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.254701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.271677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe59ad4-c753-4ff5-96cb-8460581a9b19 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.284435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.284435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.284435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.284435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] PATH_INFO: `/v3/projects/a0041e724c9d4604a4305dd31be3ae7a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.284961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0041e724c9d4604a4305dd31be3ae7a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.284961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.365879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0041e724c9d4604a4305dd31be3ae7a', 'request_id': 'req-cc3bc4b7-db41-4f70-9589-f946360d979d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:30.373113 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0041e724c9d4604a4305dd31be3ae7a', 'request_id': 'req-cc3bc4b7-db41-4f70-9589-f946360d979d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:30.381080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0041e724c9d4604a4305dd31be3ae7a', 'request_id': 'req-cc3bc4b7-db41-4f70-9589-f946360d979d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:30.393672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] The token cache is being invalidate because project a0041e724c9d4604a4305dd31be3ae7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:30.394298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:30.394614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3bc4b7-db41-4f70-9589-f946360d979d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:30.396292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1139/2239] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:30 2022] DELETE /identity/v3/projects/a0041e724c9d4604a4305dd31be3ae7a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:30.402018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.402494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.402611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.425060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:30.426323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.524413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:30.556806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1101/2240] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:30.564935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.565223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.569592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.596485 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:30.597711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:30.606354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1140/2241] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:30.610913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.626278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63011d8d-aef5-4c96-a690-0e1715b15462 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.634705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.635662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.636545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.637153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.638144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:30.638891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.642535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.666066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1102/2242] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:49:30.670888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.685147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb757cb1-b372-4c53-9678-95ee04e0a0cc None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.691975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.692968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.693348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.693713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.695343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.698191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.745928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1141/2243] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:30.750295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.766419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422318c-a73d-4c70-81c5-16e60d8870c3 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.843674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.845031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.845577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.845988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.846772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.849662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.887548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1103/2244] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:30 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:30.891871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.906442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d83064-4d52-480f-b14b-edfd4849d953 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.916030 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.916707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.916981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.917427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.918281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.920424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:30.933754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1142/2245] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:30.937845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:30.950067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b732cf-0ef7-4bc1-8361-5e80cc054966 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:30.971648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:30.972704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:30.974055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:30.977012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:30.977873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:30.983394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31086977-1feb-49ca-a6ba-9e781e5ab899 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.004487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1104/2246] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:31.009430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.025989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138d4c4-97ed-4b11-bc53-c09edccb9d6a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.041978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.043006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.043574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.043892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] PATH_INFO: `/v3/projects/df9b75372bd841babdd6d21c0861d9f3/users/3d9d67ae545e49efa7681efa3c57de17/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.087127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=df9b75372bd841babdd6d21c0861d9f3, user_id=3d9d67ae545e49efa7681efa3c57de17, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.093732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.139116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1143/2247] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:31 2022] PUT /identity/v3/projects/df9b75372bd841babdd6d21c0861d9f3/users/3d9d67ae545e49efa7681efa3c57de17/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:31.151614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.151614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.151614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.234466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] MFA Rules not processed for user `3d9d67ae545e49efa7681efa3c57de17`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:31.235734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:31.344310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1105/2248] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:31.360656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:31.360656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:31.437352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e6ae-0590-4fac-942c-1a63a370c15f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:31.441061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:31.441779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:31.442133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:31.442473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:31.448994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:31.454004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:31.455793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:31.477277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1144/2249] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:32.951279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:32.952796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:32.994377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:33.022249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002256b-6eb3-448f-b30c-d63fddfdbde7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.026671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.027323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.027648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.027928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] PATH_INFO: `/v3/users/add01791e1374db2993c4258ec6b6d6b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.029051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add01791e1374db2993c4258ec6b6d6b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.031221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.080071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add01791e1374db2993c4258ec6b6d6b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.080652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add01791e1374db2993c4258ec6b6d6b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.087354 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add01791e1374db2993c4258ec6b6d6b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.099786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add01791e1374db2993c4258ec6b6d6b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.106798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-2e99f146-904a-4d12-bb2f-3b2dbdd2ec08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add01791e1374db2993c4258ec6b6d6b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.112962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1106/2250] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:32 2022] DELETE /identity/v3/users/add01791e1374db2993c4258ec6b6d6b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:33.286779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.298411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce7066-431c-4a7f-9a39-64b01293d82f service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.355209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.355867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.356147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.356417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] PATH_INFO: `/v3/projects/4bca50d38140438ebd32cdcb25c65e3d` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.357680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bca50d38140438ebd32cdcb25c65e3d)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.359620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.405628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bca50d38140438ebd32cdcb25c65e3d', 'request_id': 'req-97d25195-9aef-44a3-b825-9deb82a32edc'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.411308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bca50d38140438ebd32cdcb25c65e3d', 'request_id': 'req-97d25195-9aef-44a3-b825-9deb82a32edc'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.419668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bca50d38140438ebd32cdcb25c65e3d', 'request_id': 'req-97d25195-9aef-44a3-b825-9deb82a32edc'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.432552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] The token cache is being invalidate because project 4bca50d38140438ebd32cdcb25c65e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:33.432841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.433487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-97d25195-9aef-44a3-b825-9deb82a32edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:33.435090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1145/2251] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:33 2022] DELETE /identity/v3/projects/4bca50d38140438ebd32cdcb25c65e3d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:33.440022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.440312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.440619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.472357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:33.473699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:33.547790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1107/2252] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:33.556109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.556472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.556824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.578122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:33.579374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:33.592308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1146/2253] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:33.597917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.611703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4a63f-bf0a-4cc9-9a92-4fc8cad1b6ed None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.616858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.617738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.618088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.618432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.619189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:33.619631 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.621477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.634429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1108/2254] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:33.640144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.650471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2978e-b969-4af6-8972-e70fcb70c258 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.655496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.656164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.656446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.656720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.657362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.659329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.689017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1147/2255] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:33.693205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.707375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d816573-0419-4204-bbcf-c8e89b4f520b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.755286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:33.773849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.774455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.774753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.775016 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.775673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.777572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.798358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1109/2256] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:33.802408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.812721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f62ada-1ca3-438f-ab2e-2b1121ae04ea admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.817268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.817860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.818134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.818399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.819029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.820670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.830393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1148/2257] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:33.835083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.845357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759a96c-ad49-4f6f-bd11-3f9800646f32 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.850774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.851584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.852011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.852283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.852975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.854658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a7fad-e26d-46ac-be48-686efb241517 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.866056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1110/2258] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:33.870370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:33.880605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77128ce0-cee8-4d79-a7fd-e69129408a9c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:33.885546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:33.886191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.886466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.886736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] PATH_INFO: `/v3/projects/d3a036ddcdf04518be6fd105bb1e6b04/users/097ba153ea614515b069fdd10a297a28/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.902148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a036ddcdf04518be6fd105bb1e6b04, user_id=097ba153ea614515b069fdd10a297a28, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:33.905329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:33.924332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1149/2259] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:33 2022] PUT /identity/v3/projects/d3a036ddcdf04518be6fd105bb1e6b04/users/097ba153ea614515b069fdd10a297a28/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:33.934475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:33.934766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:33.935043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:33.965692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] MFA Rules not processed for user `097ba153ea614515b069fdd10a297a28`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:33.966903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:34.041122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1111/2260] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:34.052004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:34.053664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:34.075596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:34.130655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f279c-9973-4550-800c-7a1013bab8f7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:34.136773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:34.137383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.137668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.137939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.145611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:34.147885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c0602-b738-4659-850d-08fe99cef5a9 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:34.156604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1150/2261] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:34.363085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:34.424256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64261342-3355-4ba4-9533-81902c7e5092 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:34.428754 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:34.429373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.429646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.429915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] PATH_INFO: `/v3/users/097ba153ea614515b069fdd10a297a28` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.431054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097ba153ea614515b069fdd10a297a28)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:34.433004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:34.482187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097ba153ea614515b069fdd10a297a28'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.482874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097ba153ea614515b069fdd10a297a28'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.490894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097ba153ea614515b069fdd10a297a28'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.492101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:34.506616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097ba153ea614515b069fdd10a297a28'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.513745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097ba153ea614515b069fdd10a297a28'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:34.521472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1112/2262] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:34 2022] DELETE /identity/v3/users/097ba153ea614515b069fdd10a297a28 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:34.526452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.526895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.527038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.546621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:34.547872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:34.610891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1151/2263] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:34.621109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:34.622518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:34.688054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9859523-a19a-4770-bb09-71e730b00b34 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:34.691763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:34.692402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:34.692677 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:34.692943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:34.703073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:34.704539 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc51c4-f022-477e-9a82-0d7fb5ce3c7e service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:34.714091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1113/2264] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:35.052856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.054343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.070513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e6cae-d80f-4cfe-b6f9-b2b132d2b248 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.087195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.087925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.088206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.090559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] PATH_INFO: `/v3/projects/d3a036ddcdf04518be6fd105bb1e6b04` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.092433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a036ddcdf04518be6fd105bb1e6b04)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.094372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.154849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a036ddcdf04518be6fd105bb1e6b04', 'request_id': 'req-9265f593-4631-4b8a-83fc-171b6003d962'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:35.160624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a036ddcdf04518be6fd105bb1e6b04', 'request_id': 'req-9265f593-4631-4b8a-83fc-171b6003d962'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:35.168976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a036ddcdf04518be6fd105bb1e6b04', 'request_id': 'req-9265f593-4631-4b8a-83fc-171b6003d962'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:35.183543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] The token cache is being invalidate because project d3a036ddcdf04518be6fd105bb1e6b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:35.183764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:35.184434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-9265f593-4631-4b8a-83fc-171b6003d962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:35.191558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1152/2265] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:35 2022] DELETE /identity/v3/projects/d3a036ddcdf04518be6fd105bb1e6b04 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:35.192056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.192506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.193547 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.215849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:35.217319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.293878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1114/2266] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:35.303842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.304144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.304593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.325718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:35.327049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.339948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1153/2267] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:35.344598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.355396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa68e38-152e-4782-9a8c-a32be9b0f65e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.360428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.361048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.361332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.361641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.362293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:35.362602 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.364282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.378353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1115/2268] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:49:35.383156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.393484 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb973cd-cc0f-472e-8ddb-57ec2be7e7f8 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.398844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.399455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.399769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.400039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.400670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.402577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.424622 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:35.435435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1154/2269] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:35.436982 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.447969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2566e3e-20c3-4822-9d98-be22c6134b04 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.514444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.515368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.515912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.516326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.517222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.519530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.529496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:35.572757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1116/2270] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:35.576887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.591211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a301e4-2a2f-43b0-bee9-8bfdfb6609d2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.603766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.604406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.604686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.604960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.605617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.607219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.622012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1155/2271] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:35.626317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.638666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd39b7-6862-457f-8efb-c9a45f9971df admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.644487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.645313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.645600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.647700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.648762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.650268 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a07f2-9d54-4cda-a0a6-45530c3dbcc9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.663048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1117/2272] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:35.667897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.680811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9b6a4-c9a7-493e-8e2f-ca15fe64ea38 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.688369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.689290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.689809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.690108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] PATH_INFO: `/v3/projects/9083d2ad151442a2b5c7b1c82afe7cf2/users/5d65b9a6a50a4097ae27bf36a137d15f/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.714430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9083d2ad151442a2b5c7b1c82afe7cf2, user_id=5d65b9a6a50a4097ae27bf36a137d15f, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.717589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:35.735947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1156/2273] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:35 2022] PUT /identity/v3/projects/9083d2ad151442a2b5c7b1c82afe7cf2/users/5d65b9a6a50a4097ae27bf36a137d15f/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:35.751443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.751762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.752034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.793352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] MFA Rules not processed for user `5d65b9a6a50a4097ae27bf36a137d15f`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:35.794589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:35.871562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1118/2274] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:35.874200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:35.983165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cadfa-8cdc-4a6e-8573-1ff4394782e8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:35.990475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:35.990475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:35.990475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:35.990475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:35.991776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:35.993982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.031833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1157/2275] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:35 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:36.032050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.052940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed266c32-fd6f-4025-9d92-a432027885ad None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.123471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.124124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.124410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.124673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.125307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.127379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.159204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1119/2276] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:36 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:36.161418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.175966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8534-87cb-44d5-948c-2f3e3e57f0fd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.181510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.182217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.182584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.182901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.183754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.185461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.198797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1158/2277] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:36.202934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.218811 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a468e-fdef-4982-82a4-6cc42209d6d1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.228556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.229352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.229801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.230104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] PATH_INFO: `/v3/projects/f8bd480ec2314d10bc61ef498047c326/users/e87c450782eb45e79f8d156782800bcc/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.249005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bd480ec2314d10bc61ef498047c326, user_id=e87c450782eb45e79f8d156782800bcc, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.252669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.289724 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1120/2278] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:36 2022] PUT /identity/v3/projects/f8bd480ec2314d10bc61ef498047c326/users/e87c450782eb45e79f8d156782800bcc/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:36.291271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.312036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81515d-cb87-4e6f-ade2-a73e40610d51 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.384629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.385181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.385459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.385873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.386368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.392314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e46fe3-1136-4668-a17b-a65ac10b4675 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.406894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1159/2279] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:36.413992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.426106 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e231-b539-4f0a-a754-33d0cd50e8da admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.432028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.432769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.433058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.433336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] PATH_INFO: `/v3/projects/f8bd480ec2314d10bc61ef498047c326/users/e87c450782eb45e79f8d156782800bcc/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.435502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bd480ec2314d10bc61ef498047c326, user_id=e87c450782eb45e79f8d156782800bcc, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.438275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.449650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1121/2280] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:36 2022] PUT /identity/v3/projects/f8bd480ec2314d10bc61ef498047c326/users/e87c450782eb45e79f8d156782800bcc/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:36.460479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b03b-9ffb-43b7-b0a0-a69b0db55ebb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.460762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b03b-9ffb-43b7-b0a0-a69b0db55ebb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.461034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74b03b-9ffb-43b7-b0a0-a69b0db55ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.491823 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ac74b03b-9ffb-43b7-b0a0-a69b0db55ebb None None] MFA Rules not processed for user `e87c450782eb45e79f8d156782800bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:36.493069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ac74b03b-9ffb-43b7-b0a0-a69b0db55ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:36.521040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:36.577886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1160/2281] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:36.587394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:36.589126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:36.613360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:36.651048 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195723b5-e0e2-4c2f-808b-872e11ee4398 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:36.654710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:36.655314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:36.655627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:36.655903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:36.664901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:36.666332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:36.676129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1122/2282] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:37.063114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29556-c1bb-4ecf-a760-5c7bcbdb00e6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.063726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29556-c1bb-4ecf-a760-5c7bcbdb00e6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.064076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29556-c1bb-4ecf-a760-5c7bcbdb00e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:37.091384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3ce29556-c1bb-4ecf-a760-5c7bcbdb00e6 None None] MFA Rules not processed for user `4aa761298e1f45e08fe4e42fba210d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:37.091384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3ce29556-c1bb-4ecf-a760-5c7bcbdb00e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:37.108119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:37.174040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1161/2283] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:37.180192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146870d-cbab-441f-874a-6644677d939e service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:37.184841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d87429-6684-4926-9732-63066f84666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:37.185477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87429-6684-4926-9732-63066f84666a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.185762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87429-6684-4926-9732-63066f84666a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.186039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87429-6684-4926-9732-63066f84666a admin admin] PATH_INFO: `/v3/users/5d65b9a6a50a4097ae27bf36a137d15f` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:37.187374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d87429-6684-4926-9732-63066f84666a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d65b9a6a50a4097ae27bf36a137d15f)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:37.189643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d87429-6684-4926-9732-63066f84666a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:37.246261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d65b9a6a50a4097ae27bf36a137d15f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.247432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d65b9a6a50a4097ae27bf36a137d15f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.256964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d65b9a6a50a4097ae27bf36a137d15f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.270339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d65b9a6a50a4097ae27bf36a137d15f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.278062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d65b9a6a50a4097ae27bf36a137d15f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.284846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1123/2284] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:37 2022] DELETE /identity/v3/users/5d65b9a6a50a4097ae27bf36a137d15f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:37.289195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.289489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.289769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:37.310778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:37.312043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:37.376736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1162/2285] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:37.386838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:37.388305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:37.440330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:37.457255 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d87429-6684-4926-9732-63066f84666a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:37.463257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:37.463988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.464335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.465103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:37.475895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:37.477568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:37.488406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1124/2286] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:37.847769 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:37.849095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:37.862012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b9286-1c80-4c0c-83bc-76ca0def1b76 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:37.867250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:37.867755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:37.868434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:37.868729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] PATH_INFO: `/v3/projects/9083d2ad151442a2b5c7b1c82afe7cf2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:37.879530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9083d2ad151442a2b5c7b1c82afe7cf2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:37.882020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:37.933127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9083d2ad151442a2b5c7b1c82afe7cf2', 'request_id': 'req-0573b4e6-5780-41a9-b36f-28ffe1f6b396'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.938743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9083d2ad151442a2b5c7b1c82afe7cf2', 'request_id': 'req-0573b4e6-5780-41a9-b36f-28ffe1f6b396'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.941344 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:37.949088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9083d2ad151442a2b5c7b1c82afe7cf2', 'request_id': 'req-0573b4e6-5780-41a9-b36f-28ffe1f6b396'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.962752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] The token cache is being invalidate because project 9083d2ad151442a2b5c7b1c82afe7cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:37.962952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.963926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:37.965661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1163/2287] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:37 2022] DELETE /identity/v3/projects/9083d2ad151442a2b5c7b1c82afe7cf2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:37.970107 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:37.980524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7e97-24ba-4634-a090-aecaf91aa644 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.040401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.041072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.041542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.041662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] PATH_INFO: `/v3/users/e87c450782eb45e79f8d156782800bcc` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.042828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87c450782eb45e79f8d156782800bcc)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.044974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.097499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87c450782eb45e79f8d156782800bcc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.097999 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87c450782eb45e79f8d156782800bcc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.105466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87c450782eb45e79f8d156782800bcc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.118761 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87c450782eb45e79f8d156782800bcc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.126341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-555a7894-20bd-4b3f-a590-15df6a288649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87c450782eb45e79f8d156782800bcc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.133332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1125/2288] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:37 2022] DELETE /identity/v3/users/e87c450782eb45e79f8d156782800bcc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:38.426916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.428914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.441832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573b4e6-5780-41a9-b36f-28ffe1f6b396 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.499829 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.500437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.500713 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.500985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] PATH_INFO: `/v3/projects/f8bd480ec2314d10bc61ef498047c326` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.502087 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bd480ec2314d10bc61ef498047c326)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.504058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.549364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bd480ec2314d10bc61ef498047c326', 'request_id': 'req-2300cd14-db21-4c16-bbf0-1dc53ca61f47'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.556276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bd480ec2314d10bc61ef498047c326', 'request_id': 'req-2300cd14-db21-4c16-bbf0-1dc53ca61f47'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.563341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bd480ec2314d10bc61ef498047c326', 'request_id': 'req-2300cd14-db21-4c16-bbf0-1dc53ca61f47'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.574524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] The token cache is being invalidate because project f8bd480ec2314d10bc61ef498047c326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:38.574808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.575443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2300cd14-db21-4c16-bbf0-1dc53ca61f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:38.576954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1164/2289] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:38 2022] DELETE /identity/v3/projects/f8bd480ec2314d10bc61ef498047c326 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:38.581915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.582379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.582514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.602633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:38.603814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.671620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1126/2290] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:38.681130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.681413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.681688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.702246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:38.703299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:38.711889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1165/2291] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:38.716442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.724033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:38.728639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4241d38-760c-4a96-956e-12ec7b71ea29 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.735156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.735815 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.736096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.736617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.737272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:38.737737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.739537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.754545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1127/2292] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:49:38.759081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.768605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f72d9-4b3b-4e95-a920-18a3ba0829f6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.773307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.773894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.774177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.774520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.775095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.777039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.806866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1166/2293] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:38.811062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.821457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307b9f-ab03-4f8e-a6b8-281e8e28726d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.879042 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.879925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.880203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.880471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.881103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.883012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.907908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1128/2294] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:38 2022] POST /identity/v3/users => generated 278 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:38.909377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.920351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa59fb8-df9a-474f-9eb6-72973f0775ec admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.925043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.925642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.925929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.926207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.926876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.928538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.941718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1167/2295] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:38.942968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.953090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9747e-34f2-45aa-a555-7a986d7b35ac admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.958102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.958691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.958969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.959797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:38.960470 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:38.962074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117bd566-b170-4fbf-897a-f30be1108582 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:38.973160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1129/2296] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:38.978096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:38.988238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce2d76-a7a2-48b0-b980-12fc2c7f500b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:38.992866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:38.993502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:38.993777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:38.994047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] PATH_INFO: `/v3/projects/3d3957eed9534f5a84508a124e76439d/users/5d90cfd9e9e44805952865dd7b9052c9/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.008304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3957eed9534f5a84508a124e76439d, user_id=5d90cfd9e9e44805952865dd7b9052c9, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.010939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.023907 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1168/2297] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:38 2022] PUT /identity/v3/projects/3d3957eed9534f5a84508a124e76439d/users/5d90cfd9e9e44805952865dd7b9052c9/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:39.034952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.035276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.035648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.064767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] MFA Rules not processed for user `5d90cfd9e9e44805952865dd7b9052c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:39.065976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:39.129369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1130/2298] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:39.134044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.151850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:39.196787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54350b63-737e-4ec7-8e80-062aeb68df7d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.201454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.202052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.202334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.202608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.203240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.205184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.231662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1169/2299] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:39.235820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.246744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812bc62-2d9b-47b0-b25a-1b22abcb2e94 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.313331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.313951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.314241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.314512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.315139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.317113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.328304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:39.348213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1131/2300] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:39.352276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.364480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7d1c-d350-4129-aea7-629e61684559 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.370598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.371219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.371524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.371811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.372793 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.374456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.389846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1170/2301] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:39.391003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.405521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385bc8-a1e9-492b-ba11-1b0aa81c8fc1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.412219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.413067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.413227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.413472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] PATH_INFO: `/v3/projects/d7f0329cd1c14d2fa077e63675335134/users/cf0c2c0b2fd14d78b0e2fd4800a416da/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.432168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0329cd1c14d2fa077e63675335134, user_id=cf0c2c0b2fd14d78b0e2fd4800a416da, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.439381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.453382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1132/2302] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:39 2022] PUT /identity/v3/projects/d7f0329cd1c14d2fa077e63675335134/users/cf0c2c0b2fd14d78b0e2fd4800a416da/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:49:39.457508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.469709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be061-e378-4356-8520-e57db6754b12 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.528672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.529294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.529573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.529906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.530546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.532187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d384ae-7669-4124-9abd-797cb94f7e66 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.542945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1171/2303] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:39.547392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.558308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbfe89-ec76-4117-bd00-82e6cce409c4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.562996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.563807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.564083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.564353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] PATH_INFO: `/v3/projects/d7f0329cd1c14d2fa077e63675335134/users/cf0c2c0b2fd14d78b0e2fd4800a416da/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.566661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0329cd1c14d2fa077e63675335134, user_id=cf0c2c0b2fd14d78b0e2fd4800a416da, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.569378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.580038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1133/2304] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:39 2022] PUT /identity/v3/projects/d7f0329cd1c14d2fa077e63675335134/users/cf0c2c0b2fd14d78b0e2fd4800a416da/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:39.591695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.592241 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.592515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.620348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] MFA Rules not processed for user `cf0c2c0b2fd14d78b0e2fd4800a416da`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:39.621568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:39.691137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1172/2305] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:39.702671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.705071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:39.769919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2940f-0dde-4d02-84e4-deeb9f544669 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.777148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.777960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.778653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.778976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.792168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.793611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.804178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1134/2306] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:39.894868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:39.907154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138f5a-46c9-46f7-9c8c-4262c4eb0eec None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:39.912025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:39.914080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:39.914554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:39.915053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:39.924891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:39.985659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:39.987116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4582c-3f85-467b-8f31-f2dd8e435e9e service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:39.999805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1173/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:47.412654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.414329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:47.539806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddafe5a-3707-4d6f-a42b-f78a1bc67bb5 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.545604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.546294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.546658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.547067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] PATH_INFO: `/v3/users/0f1b612f1d3b45748c063509c48f65a0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.548533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1b612f1d3b45748c063509c48f65a0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:47.551298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:47.646164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1b612f1d3b45748c063509c48f65a0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:47.647943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1b612f1d3b45748c063509c48f65a0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:47.667703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1b612f1d3b45748c063509c48f65a0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:47.671666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:47.697891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1b612f1d3b45748c063509c48f65a0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:47.711218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1b612f1d3b45748c063509c48f65a0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:47.725190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1135/2308] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:47 2022] DELETE /identity/v3/users/0f1b612f1d3b45748c063509c48f65a0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:47.729388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.729867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.730064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:47.758617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:47.760040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:47.864646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1174/2309] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:47.878537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:47.880713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:47.981416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459cc7e3-880a-439c-b8c1-d2bf80fd0e2b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:47.988160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:47.989317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:47.990196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:47.990619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.010693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.012577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff47e9a-38c4-4f4b-a086-1bb5098884e1 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.029339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1136/2310] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:48.332493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.335267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:48.354691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec162585-ec52-4ec1-a629-20a6fdcd02cd None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.359981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.360701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.361341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.361809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] PATH_INFO: `/v3/projects/7e1158cd7f24450abfa06ee83a159c7d` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.363009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1158cd7f24450abfa06ee83a159c7d)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.365190 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.426286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1158cd7f24450abfa06ee83a159c7d', 'request_id': 'req-a4045760-22b2-44c3-bb5c-6ec9675e60d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:48.434922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1158cd7f24450abfa06ee83a159c7d', 'request_id': 'req-a4045760-22b2-44c3-bb5c-6ec9675e60d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:48.449210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1158cd7f24450abfa06ee83a159c7d', 'request_id': 'req-a4045760-22b2-44c3-bb5c-6ec9675e60d3'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:48.463922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] The token cache is being invalidate because project 7e1158cd7f24450abfa06ee83a159c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:48.464202 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:48.464843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a4045760-22b2-44c3-bb5c-6ec9675e60d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:48.466334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1175/2311] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:48 2022] DELETE /identity/v3/projects/7e1158cd7f24450abfa06ee83a159c7d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:48.472383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.472961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.472961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.495799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:48.497030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:48.557391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1137/2312] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:48.565295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.565587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.565863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.586332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:48.587572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:48.596237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1176/2313] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:48.600572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.612296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af9b05-ca10-4ec5-9a23-a67658ac2aff None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.617440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.618057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.618337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.618609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.619327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:48.619667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.621370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.635179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1138/2314] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:48.639808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.650561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a70614-84a1-4089-b1b9-208c5291314c None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.655669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.656277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.656555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.656975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.657461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.659509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.690213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1177/2315] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:48 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:48.694995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.705367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd65100-6740-4e83-aef4-72e82190dd22 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.765340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.765986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.766445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.766722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.767374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.769377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.780781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:48.799770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1139/2316] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:48 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:48.802125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.811136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:48.816049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3018de-5aab-4714-b0da-f53761741512 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.825585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.826196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.826537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.826810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.827462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.829223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.840253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1178/2317] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:48.844624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.855498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3168a8-8ee7-4c51-9e71-12ef75682951 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.861964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.862871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.863159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.863436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.864188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.866128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6301e7-8c90-4dd0-b0cf-732819d24ad7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.879267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1140/2318] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:48.889532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:48.902983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3ef3a-3044-4ac0-819a-bb8ba5cddb57 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:48.907935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:48.908695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.908988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.909381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] PATH_INFO: `/v3/projects/c88aa1104bc6412a9b604de7db1bbdb8/users/3637a2b86e4d44e2beaf692307447bd8/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:48.936378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88aa1104bc6412a9b604de7db1bbdb8, user_id=3637a2b86e4d44e2beaf692307447bd8, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:48.939060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:48.955038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1179/2319] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:48 2022] PUT /identity/v3/projects/c88aa1104bc6412a9b604de7db1bbdb8/users/3637a2b86e4d44e2beaf692307447bd8/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:48.968013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00551-9d25-4828-8adb-6d10e2007fed None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:48.968305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00551-9d25-4828-8adb-6d10e2007fed None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:48.968583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd00551-9d25-4828-8adb-6d10e2007fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:49.012923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-afd00551-9d25-4828-8adb-6d10e2007fed None None] MFA Rules not processed for user `3637a2b86e4d44e2beaf692307447bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:49.014177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-afd00551-9d25-4828-8adb-6d10e2007fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:49.114623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1141/2320] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:49.146058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:49.148231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:49.228229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2a4be-fac5-4271-90d1-d12b97fe5176 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:49.232072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:49.232757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:49.233122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:49.233468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:49.243000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:49.246860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8634cf3-020b-40b8-b406-de058f73d9cd service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:49.254123 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1180/2321] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:50.204298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:50.204817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:50.204951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:50.206239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1142/2322] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:49:50.225449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:50.226501 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:50.226596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:50.245628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] MFA Rules not processed for user `cb029fc999c644408f292f85e4ab114b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:50.246844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:50.313696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1181/2323] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:49:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:50.335583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:50.345181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:50.365277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b8636-3efe-41b0-848e-0bfdeddaaa80 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:50.371941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:50.372774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:50.373257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:50.373716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:50.385840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:50.387694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:50.411654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1143/2324] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:51.068820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:51.070299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:51.134426 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee937b04-d98b-41c9-8840-3c6874574939 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:51.138916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:51.139617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:51.139895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:51.140165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] PATH_INFO: `/v3/users/9438dde54dda4baab42a5fc448dfb7f8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:51.141281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9438dde54dda4baab42a5fc448dfb7f8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:51.143214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:51.155195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:51.192179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9438dde54dda4baab42a5fc448dfb7f8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.193733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9438dde54dda4baab42a5fc448dfb7f8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.202105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9438dde54dda4baab42a5fc448dfb7f8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.209188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9438dde54dda4baab42a5fc448dfb7f8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.221846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9438dde54dda4baab42a5fc448dfb7f8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.227751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1182/2325] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:51 2022] DELETE /identity/v3/users/9438dde54dda4baab42a5fc448dfb7f8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:51.400717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:51.412985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee144fe-c7e9-41bd-ba86-4edd944dce51 service placement] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:51.473208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:51.473839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:51.474118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:51.474391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] PATH_INFO: `/v3/projects/e164e46e8f3643babea969dd5e4d08f5` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:51.475558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e164e46e8f3643babea969dd5e4d08f5)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:51.477500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:51.536240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e164e46e8f3643babea969dd5e4d08f5', 'request_id': 'req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.541818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e164e46e8f3643babea969dd5e4d08f5', 'request_id': 'req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.544348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:51.549949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e164e46e8f3643babea969dd5e4d08f5', 'request_id': 'req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.563069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] The token cache is being invalidate because project e164e46e8f3643babea969dd5e4d08f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:51.563351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.564026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.565515 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1144/2326] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:51 2022] DELETE /identity/v3/projects/e164e46e8f3643babea969dd5e4d08f5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:51.570039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:51.571440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:51.584352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf02db-e21f-4d52-915b-fd736bff25d2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:51.648754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:51.649465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:51.649771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:51.650022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] PATH_INFO: `/v3/users/61117f822d0a4e39a6443e77741898b1` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:51.651305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=61117f822d0a4e39a6443e77741898b1)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:51.653354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:51.713619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61117f822d0a4e39a6443e77741898b1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.714276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61117f822d0a4e39a6443e77741898b1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.721952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61117f822d0a4e39a6443e77741898b1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.729502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61117f822d0a4e39a6443e77741898b1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.741815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-312d4b51-47cd-4a2c-beb1-ee2b39f543cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61117f822d0a4e39a6443e77741898b1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:51.747982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1183/2327] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:51 2022] DELETE /identity/v3/users/61117f822d0a4e39a6443e77741898b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:51.932302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:51.934088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:51.948749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56920c-ca05-4bdf-9e2f-06c5fa0f7a63 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.044577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.045789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.046141 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.047281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] PATH_INFO: `/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.048793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc19f008f1ea4723913ee7de9d3a7b84)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.050722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.107312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc19f008f1ea4723913ee7de9d3a7b84', 'request_id': 'req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:52.117450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc19f008f1ea4723913ee7de9d3a7b84', 'request_id': 'req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:52.127423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc19f008f1ea4723913ee7de9d3a7b84', 'request_id': 'req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:52.141063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] The token cache is being invalidate because project cc19f008f1ea4723913ee7de9d3a7b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:52.141352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:52.142002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7aefbe2e-7cc2-490d-aafe-57ee9d4b6238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:52.143519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1145/2328] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:51 2022] DELETE /identity/v3/projects/cc19f008f1ea4723913ee7de9d3a7b84 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:52.149395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.149619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.149926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.165969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:52.172354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:52.174416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:52.243891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1184/2329] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:52.252087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01be346-de9c-4115-a900-a70b801a745c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.252459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01be346-de9c-4115-a900-a70b801a745c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.252736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01be346-de9c-4115-a900-a70b801a745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.278180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b01be346-de9c-4115-a900-a70b801a745c None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:52.279578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b01be346-de9c-4115-a900-a70b801a745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:52.288901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1146/2330] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:52.292917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.303818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b20fe38-aacb-4ee7-921f-1c00db86e303 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.309825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.310644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.310917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.311193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.311881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:52.312182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.314311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.327766 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1185/2331] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:52.332159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01be346-de9c-4115-a900-a70b801a745c None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.342679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01be346-de9c-4115-a900-a70b801a745c None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.347467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.348093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.348391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.348664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.349290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.352047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.381000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1147/2332] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:52.385539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.396598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a724-2613-4e43-aa39-62cb4de02ea3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.453728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.454358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.454621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.455146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.455537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.457689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.481216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1186/2333] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:52.485575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.497115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a468311-f2bc-4380-827a-f895e542bfc5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.502904 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.503440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.503756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.504017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.504664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.506273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.516806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1148/2334] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:52.521020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.532850 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6fbd1-5a7e-43ff-9a25-95b2127269bc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.538883 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.539631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.539988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.540290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.541067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.542742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc136367-8012-4896-88a8-57318a2ac8d5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.555243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1187/2335] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:52.560825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.573291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36c5bc-bf8e-447b-bb9a-0e08fe67fe9d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.578219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.578879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.579160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.579431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] PATH_INFO: `/v3/projects/d836437226dc4eb485ba02cc264fb072/users/221d5130322d4d99bcf1fda9e2c79856/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.600252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d836437226dc4eb485ba02cc264fb072, user_id=221d5130322d4d99bcf1fda9e2c79856, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.604325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.623124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1149/2336] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:52 2022] PUT /identity/v3/projects/d836437226dc4eb485ba02cc264fb072/users/221d5130322d4d99bcf1fda9e2c79856/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:52.634623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.634906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.635177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.663086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] MFA Rules not processed for user `221d5130322d4d99bcf1fda9e2c79856`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:52.664869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:52.741133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1188/2337] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:52.747851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.770067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:52.833374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ca8e-b2df-497a-a241-a0bff4770f4d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:52.843252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:52.844215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:52.844632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:52.845009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:52.846033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:52.848057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:52.888301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1150/2338] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:52.890246 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:52.903542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:52.909028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101c9a-f77c-4021-b0e0-23de32873008 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.004754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.005369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.005653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.005928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.006641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.009396 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.033651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1189/2339] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:52 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:53.037907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:53.051778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139e6f-ddbf-4733-bd20-914a3d3104b5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.057483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.058160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.058501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.058773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.059414 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.061116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.073399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1151/2340] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:53.077770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:53.092476 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d782d-745e-4b6f-90bc-e05f8b14b9e2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.096292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.096292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.096292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.096292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] PATH_INFO: `/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e/users/f7e4a23f1d9a493792cbc04887b7c0c7/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.112461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3243693955cc4d2fa97c70ee4bc9f56e, user_id=f7e4a23f1d9a493792cbc04887b7c0c7, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.115236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.128689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1190/2341] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:53 2022] PUT /identity/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e/users/f7e4a23f1d9a493792cbc04887b7c0c7/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:53.132701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:53.144152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5450393-4c2d-4b8c-9807-e47b2b834e61 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.223467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.224053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.224330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.224600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.225391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.227758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408359c7-7244-4ce8-b80e-a688b1a1d578 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.240476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1152/2342] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:53.245903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:53.257100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a4c26-6403-46bf-aff9-a54c2d70e39c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.263567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.264251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.264540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.264886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] PATH_INFO: `/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e/users/f7e4a23f1d9a493792cbc04887b7c0c7/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.266938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3243693955cc4d2fa97c70ee4bc9f56e, user_id=f7e4a23f1d9a493792cbc04887b7c0c7, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.269688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.282901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1191/2343] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:53 2022] PUT /identity/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e/users/f7e4a23f1d9a493792cbc04887b7c0c7/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:53.353603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.353894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.354187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.386343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] MFA Rules not processed for user `f7e4a23f1d9a493792cbc04887b7c0c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:53.387601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:53.464827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1153/2344] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:53.476046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:53.477746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:53.542172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b4f1c-7700-476e-97f5-a481e0e90e0d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:53.546139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:53.546745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:53.547018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:53.547283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:53.604861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:53.606306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064a4c-4b52-4d54-b5ad-a19cd91cd0c9 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:53.615178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1192/2345] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:57.944767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:57.946455 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:57.960571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:58.014586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae33e0e-fc26-4db6-b63a-26b63dfadeec None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.019159 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.019823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.020104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.020376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] PATH_INFO: `/v3/users/3b8996a317c3417bbb4618d4fee1c668` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.021535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b8996a317c3417bbb4618d4fee1c668)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.023642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.074506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b8996a317c3417bbb4618d4fee1c668'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.075100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b8996a317c3417bbb4618d4fee1c668'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.083717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b8996a317c3417bbb4618d4fee1c668'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.097491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b8996a317c3417bbb4618d4fee1c668'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.104632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b8996a317c3417bbb4618d4fee1c668'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.110720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1154/2346] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:49:57 2022] DELETE /identity/v3/users/3b8996a317c3417bbb4618d4fee1c668 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:58.115401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.115970 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.116135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.135727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:58.137044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.141157 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:58.200645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1193/2347] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:58.213075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.214491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.272030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20bf6-6268-489c-8b64-599586c3d7f9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.276293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.276894 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.277171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.277440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.286340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.287914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7227e-7f26-464f-8ad2-b89ad4101bae service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.297224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1155/2348] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:49:58.562472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.564059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.576200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722abb-99af-44e3-aa75-e4394cc6cb33 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.580913 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.581621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.581926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.582168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] PATH_INFO: `/v3/projects/473ca8b81f084f81baee856774f57eb3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.583298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=473ca8b81f084f81baee856774f57eb3)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.585869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.634335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473ca8b81f084f81baee856774f57eb3', 'request_id': 'req-eaef1932-78f5-44da-b396-41ae4ecc29f9'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.640299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473ca8b81f084f81baee856774f57eb3', 'request_id': 'req-eaef1932-78f5-44da-b396-41ae4ecc29f9'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.649187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473ca8b81f084f81baee856774f57eb3', 'request_id': 'req-eaef1932-78f5-44da-b396-41ae4ecc29f9'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.660600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] The token cache is being invalidate because project 473ca8b81f084f81baee856774f57eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:49:58.660892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.661534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-eaef1932-78f5-44da-b396-41ae4ecc29f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:49:58.663772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1194/2349] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:49:58 2022] DELETE /identity/v3/projects/473ca8b81f084f81baee856774f57eb3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:58.668529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.668821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.669102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.690083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:58.691293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.723658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:58.754453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1156/2350] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:58.762941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.763250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.763561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.783989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:58.785371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:58.793372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1195/2351] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:49:58.797623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.807780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2f9a2-a3f1-472f-9af3-37dee38972f1 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.812487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.813092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.813374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.813647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.814288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:49:58.814591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.816254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.828285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1157/2352] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:49:58.832705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.842840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25594d59-d42e-4c5a-9e2f-546bbb9bded1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.847643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.848401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.848688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.848967 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.849595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.852116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.880058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1196/2353] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:49:58.886083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.896445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee9638-4a9d-4942-a0a4-5cfd5d3f2da9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:58.953134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:58.953742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:58.954018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:58.954285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:58.954895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:58.956850 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:58.983673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1158/2354] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:49:58 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:49:58.987647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:58.999435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33948a-b8f0-4a52-a961-5c9f7783bee8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.005597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.006322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.006711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.007114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.008072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.010080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.021016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1197/2355] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:49:59.027043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.038607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cbe51-1450-4889-af90-02c1b04a7f16 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.044484 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.045092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.045371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.045643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.046338 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.047986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37996734-f716-47f0-aef8-87538a69ee0d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.060301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1159/2356] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:49:59.064438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.074313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bd5b4-4670-4199-8c24-e99b8e15e9b8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.078993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.079678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.079957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.080232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] PATH_INFO: `/v3/projects/b3b756a7bd444bf3aec562b2717f5623/users/21fdd2fd30544e4e9e0bf184034106fa/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.095318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3b756a7bd444bf3aec562b2717f5623, user_id=21fdd2fd30544e4e9e0bf184034106fa, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.098008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.111699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1198/2357] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:49:59 2022] PUT /identity/v3/projects/b3b756a7bd444bf3aec562b2717f5623/users/21fdd2fd30544e4e9e0bf184034106fa/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:49:59.116908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211f038-d182-47cd-8fbe-862e09428836 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.117215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211f038-d182-47cd-8fbe-862e09428836 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.117486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211f038-d182-47cd-8fbe-862e09428836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.136347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f211f038-d182-47cd-8fbe-862e09428836 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:59.137489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f211f038-d182-47cd-8fbe-862e09428836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:59.195356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1160/2358] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:49:59.205196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:49:59.206918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:59.240825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:59.267776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619f8bf-b7d3-4564-92db-c88245c831cd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:49:59.271550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:49:59.272734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.273074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.273410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.282986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:49:59.284564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:49:59.295129 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1199/2359] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:49:59.891336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:49:59.891800 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:49:59.892174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:49:59.921605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] MFA Rules not processed for user `21fdd2fd30544e4e9e0bf184034106fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:49:59.922917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:49:59.953244 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:49:59.991752 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1161/2360] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:00.002298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:00.003820 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:00.092382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a75903-90bb-4aba-b991-f712df29bb77 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:00.096343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:00.097236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:00.097522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:00.097895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:00.108204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:00.109649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93289c-cfa6-4d99-94a5-3e2bc57f5da7 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:00.118706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1200/2361] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:01.222256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.223776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.237994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaedd811-60fe-4e48-aff5-c03bfccf40ac None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.243373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.244189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.244535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.244877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] PATH_INFO: `/v3/users/3d9d67ae545e49efa7681efa3c57de17` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.246082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9d67ae545e49efa7681efa3c57de17)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.248045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.296434 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9d67ae545e49efa7681efa3c57de17'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.297203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9d67ae545e49efa7681efa3c57de17'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.305891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9d67ae545e49efa7681efa3c57de17'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.318335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9d67ae545e49efa7681efa3c57de17'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.325202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9d67ae545e49efa7681efa3c57de17'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:01.331739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1162/2362] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:01 2022] DELETE /identity/v3/users/3d9d67ae545e49efa7681efa3c57de17 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:01.336937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94a5f-8c07-445b-b8cc-8545312456c8 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.337431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94a5f-8c07-445b-b8cc-8545312456c8 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.337550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94a5f-8c07-445b-b8cc-8545312456c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.356933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8ee94a5f-8c07-445b-b8cc-8545312456c8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:01.358077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8ee94a5f-8c07-445b-b8cc-8545312456c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.418088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1201/2363] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:01.427928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.429367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.476180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a385071-fba0-49b2-8766-69923330a046 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.476465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a385071-fba0-49b2-8766-69923330a046 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.476743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a385071-fba0-49b2-8766-69923330a046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.488247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f755-58a5-473a-a41e-424273dbe53b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.492126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.492773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.493056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.493328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.496616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4a385071-fba0-49b2-8766-69923330a046 None None] MFA Rules not processed for user `21fdd2fd30544e4e9e0bf184034106fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:01.497826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4a385071-fba0-49b2-8766-69923330a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.502957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.504438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:01.514281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1163/2364] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:01.562566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1202/2365] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:01.948325 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:01.950373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:01.977422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dd12b-c212-4156-8125-c4a353652b36 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:01.985343 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:01.986028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:01.986359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:01.986718 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] PATH_INFO: `/v3/projects/df9b75372bd841babdd6d21c0861d9f3` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:01.988704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df9b75372bd841babdd6d21c0861d9f3)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:01.992428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.043471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:02.056625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df9b75372bd841babdd6d21c0861d9f3', 'request_id': 'req-bfe84ece-b512-4c9c-8b09-2f768d222218'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:02.063037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df9b75372bd841babdd6d21c0861d9f3', 'request_id': 'req-bfe84ece-b512-4c9c-8b09-2f768d222218'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:02.072645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df9b75372bd841babdd6d21c0861d9f3', 'request_id': 'req-bfe84ece-b512-4c9c-8b09-2f768d222218'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:02.089372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] The token cache is being invalidate because project df9b75372bd841babdd6d21c0861d9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:02.089674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:02.090473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-bfe84ece-b512-4c9c-8b09-2f768d222218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:02.092770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1164/2366] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:01 2022] DELETE /identity/v3/projects/df9b75372bd841babdd6d21c0861d9f3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:02.098532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.098693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.098963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.125303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:02.126814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.143644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:02.205442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1203/2367] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:02.226293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.226293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.226293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.259984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:02.261460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.301755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1165/2368] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:02.303091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.317008 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4c08a-9ef7-4422-ad62-1512400f9ca6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.326952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.328297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.328297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.328750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.329425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:02.329862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.336203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.362608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1204/2369] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:02.364021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.383819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91f362-1adf-47b9-a8dc-41c37ca768a3 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.383819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.383819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.383819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.384183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.384840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.387098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.443684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1166/2370] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:02.447023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.459052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2d517-e215-4ccd-9bf6-6303a6651c9c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.548857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.548857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.548857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.548857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.549715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.551959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.585574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1205/2371] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:02.589516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.612533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a750-1d74-44ce-bf3c-841e4c5c04d9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.619932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.620772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.621550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.623321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.624262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.625855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.637564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1167/2372] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:02.641728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.658158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26da8c-6790-40c1-8b05-a1e5306cd30d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.664555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.665254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.665612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.666076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.667081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.668734 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d299e-b2a5-4ef8-abb3-2dcf444091d2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.684045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1206/2373] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:02.688190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.700997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f07ca-b589-4993-bb50-63b229dca033 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.706401 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.707161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.707544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.708023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] PATH_INFO: `/v3/projects/2bf760930ff8494e8c7e75cdd496dd23/users/f1423be91ecd462c95ea57058eaee62e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.728195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf760930ff8494e8c7e75cdd496dd23, user_id=f1423be91ecd462c95ea57058eaee62e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.730978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.747644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1168/2374] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:02 2022] PUT /identity/v3/projects/2bf760930ff8494e8c7e75cdd496dd23/users/f1423be91ecd462c95ea57058eaee62e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:02.760247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.760656 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.761009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.790334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] MFA Rules not processed for user `f1423be91ecd462c95ea57058eaee62e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:02.791601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.869472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1207/2375] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:02.881093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:02.883303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:02.942113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09f3ab-22fe-4f3c-8338-d195da6265cb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:02.946101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:02.946742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:02.947024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:02.947299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:02.958165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:02.960156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:02.969431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1169/2376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:05.545186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:05.563146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8eb4f-3038-4a71-a2d8-a689520bc4b3 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:05.575799 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:05.576575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:05.577230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:05.577230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:05.582106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:05.619083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:05.757874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:05.759373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:05.774243 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1208/2377] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:06.561803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:06.564179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:06.701230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d117c-2e18-4ee7-9097-b217418d3f21 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:06.711753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:06.712865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:06.713335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:06.713791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] PATH_INFO: `/v3/users/5d90cfd9e9e44805952865dd7b9052c9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:06.715554 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d90cfd9e9e44805952865dd7b9052c9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:06.719128 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:06.731471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:06.732227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:06.777004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad8388-5517-4b65-8ebb-bcb544399418 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:06.783165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:06.784261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:06.785399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:06.785518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] PATH_INFO: `/v3/users/f1423be91ecd462c95ea57058eaee62e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:06.789667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1423be91ecd462c95ea57058eaee62e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:06.792421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:06.806777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d90cfd9e9e44805952865dd7b9052c9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.808909 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d90cfd9e9e44805952865dd7b9052c9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.828890 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d90cfd9e9e44805952865dd7b9052c9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.852499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d90cfd9e9e44805952865dd7b9052c9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.861949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-310f8149-2a1a-4523-8a6b-f70f1bfc7e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d90cfd9e9e44805952865dd7b9052c9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.875813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1170/2378] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:06 2022] DELETE /identity/v3/users/5d90cfd9e9e44805952865dd7b9052c9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:06.880942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:06.882023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:06.882305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:06.912110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1423be91ecd462c95ea57058eaee62e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.913640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1423be91ecd462c95ea57058eaee62e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.924764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:06.927781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:06.930147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1423be91ecd462c95ea57058eaee62e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.943401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1423be91ecd462c95ea57058eaee62e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:06.989709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-3ab32c79-aa7e-46cf-b4a0-fcedcf04c148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1423be91ecd462c95ea57058eaee62e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:07.001802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1209/2379] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:06 2022] DELETE /identity/v3/users/f1423be91ecd462c95ea57058eaee62e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:07.010794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.011415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.011918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.064763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:07.066166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.082895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1171/2380] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:07.089929 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.092971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.145364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1210/2381] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:07.154502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.155964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.178882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94aaaa-b714-4cb8-8639-37cdab76af0a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.185338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:07.218418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5eb04-7b1a-46e1-87f9-91d1cba9e5bb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.220712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:07.222373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.223137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.223654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.223906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.232797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.234080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.234667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.235275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.236491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.238026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.248899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.250832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:07.253442 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1211/2382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:07.262081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1172/2383] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:07.850423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:07.854948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:07.919684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf6193-d1f7-4b6f-92be-5be20e1be520 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:07.932420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:07.938945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:07.939234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:07.939893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] PATH_INFO: `/v3/projects/2bf760930ff8494e8c7e75cdd496dd23` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:07.941266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf760930ff8494e8c7e75cdd496dd23)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:07.944538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.017082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf760930ff8494e8c7e75cdd496dd23', 'request_id': 'req-4d06c83d-323a-436f-b87c-fddd233ba898'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.027169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf760930ff8494e8c7e75cdd496dd23', 'request_id': 'req-4d06c83d-323a-436f-b87c-fddd233ba898'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.047390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf760930ff8494e8c7e75cdd496dd23', 'request_id': 'req-4d06c83d-323a-436f-b87c-fddd233ba898'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.081086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] The token cache is being invalidate because project 2bf760930ff8494e8c7e75cdd496dd23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:08.081422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.082177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.084663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1212/2384] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:07 2022] DELETE /identity/v3/projects/2bf760930ff8494e8c7e75cdd496dd23 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:08.214342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.219241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.254173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefbf8d-2ddc-4232-aa17-cd08382d8f24 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.392238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.393099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.393449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.393796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] PATH_INFO: `/v3/projects/3d3957eed9534f5a84508a124e76439d` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.395224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3957eed9534f5a84508a124e76439d)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.398131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.460578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3957eed9534f5a84508a124e76439d', 'request_id': 'req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.470222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3957eed9534f5a84508a124e76439d', 'request_id': 'req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.484798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3957eed9534f5a84508a124e76439d', 'request_id': 'req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.504860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] The token cache is being invalidate because project 3d3957eed9534f5a84508a124e76439d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:08.505368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.505848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.508004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1173/2385] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:08 2022] DELETE /identity/v3/projects/3d3957eed9534f5a84508a124e76439d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:08.513019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.514727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.536130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06c83d-323a-436f-b87c-fddd233ba898 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:08.607555 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:08.608206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:08.608487 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:08.608759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] PATH_INFO: `/v3/users/cf0c2c0b2fd14d78b0e2fd4800a416da` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:08.609896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0c2c0b2fd14d78b0e2fd4800a416da)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:08.611893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:08.661549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0c2c0b2fd14d78b0e2fd4800a416da'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.662728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0c2c0b2fd14d78b0e2fd4800a416da'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.665420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:08.671955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0c2c0b2fd14d78b0e2fd4800a416da'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.681386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0c2c0b2fd14d78b0e2fd4800a416da'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.693313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8017049a-5194-4af1-9e61-92b110cd6abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0c2c0b2fd14d78b0e2fd4800a416da'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:08.699268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1213/2386] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:08 2022] DELETE /identity/v3/users/cf0c2c0b2fd14d78b0e2fd4800a416da => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:08.968110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:08.969640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:08.984747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a57b4c-9996-4d73-ab8c-adbe75ff0acc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.047004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.047715 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.048018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.048420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] PATH_INFO: `/v3/projects/d7f0329cd1c14d2fa077e63675335134` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.049743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f0329cd1c14d2fa077e63675335134)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.051799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.070273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:09.103765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f0329cd1c14d2fa077e63675335134', 'request_id': 'req-6faa621e-809a-442d-ae44-fee74d86e424'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.109760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f0329cd1c14d2fa077e63675335134', 'request_id': 'req-6faa621e-809a-442d-ae44-fee74d86e424'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.116902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f0329cd1c14d2fa077e63675335134', 'request_id': 'req-6faa621e-809a-442d-ae44-fee74d86e424'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.128113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] The token cache is being invalidate because project d7f0329cd1c14d2fa077e63675335134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:09.128400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.129021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6faa621e-809a-442d-ae44-fee74d86e424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:09.130543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1174/2387] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:08 2022] DELETE /identity/v3/projects/d7f0329cd1c14d2fa077e63675335134 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:09.135446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.135955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.136060 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.155832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:09.157041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.216481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1214/2388] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:09.224341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.225216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.225216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.245818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:09.246963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.254916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1175/2389] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:09.259695 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.269463 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f9a11-0348-4b88-9d78-5888a5331ad1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.274741 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.275336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.275649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.275924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.276568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:09.276871 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.278470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.289787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1215/2390] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:09.293865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.303158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18964bc-2ce7-4322-acf1-89810343e28a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.307885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.308477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.308756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.309027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.309645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.311585 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.342565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1176/2391] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:09.354079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.364220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499188-c246-4306-9ee5-ee3909a01f2d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.438281 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.438792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.439290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.439597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.440363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.444664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.476262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1216/2392] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:09.478097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.489127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08e428-f70d-4415-96a3-ac22f0329625 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.494616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.495236 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.495545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.495878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.496487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.498094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.514096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1177/2393] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:09.519939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.533988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcc99e-2c1c-4a22-9e82-e414398b3ab4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.539427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.540171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.540446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.540716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.541357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.542951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7993f-3200-4794-ad11-20c6c00386b4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.555257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1217/2394] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:09.559395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.570474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a083ae-46f0-4d26-9a9a-f40792f60b1f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.575329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.576019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.576298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.576884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] PATH_INFO: `/v3/projects/5a2e8c4ef81a43f18138efd02bf96832/users/932fca992ede41479e90827e95b4387f/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.593694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2e8c4ef81a43f18138efd02bf96832, user_id=932fca992ede41479e90827e95b4387f, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.596737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.610673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1178/2395] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:09 2022] PUT /identity/v3/projects/5a2e8c4ef81a43f18138efd02bf96832/users/932fca992ede41479e90827e95b4387f/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:09.621541 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.621810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.622083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.650447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] MFA Rules not processed for user `932fca992ede41479e90827e95b4387f`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:09.656916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:09.677226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:09.725874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1218/2396] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:09.730402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.791881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed17c6c-031f-4936-8070-37e80f30c2a1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.796459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.797052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.797328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.797597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.798217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.800172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.826833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1179/2397] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:09.830899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.841958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0e19-9516-4808-ad4a-973dd7b3bba2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.897819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.898423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.898702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.898977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.899647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.901574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.922432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1219/2398] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:09 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:09.926478 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.937490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72482dfa-958f-48d2-b7d9-03a2ff080bdf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.942378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.942980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.943254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.944174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.944843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.946472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:09.957002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1180/2399] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:09.961057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:09.971187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3aacb-3176-4e2e-8fb0-93aba516e5cb admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:09.976353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:09.977002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:09.977280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:09.977552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] PATH_INFO: `/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8/users/d61fd993f4a643a49d773d1a7cea11fa/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:09.992920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b94c53bb2b34b84b2dddf374044f2e8, user_id=d61fd993f4a643a49d773d1a7cea11fa, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:09.995681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.008272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1220/2400] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:09 2022] PUT /identity/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8/users/d61fd993f4a643a49d773d1a7cea11fa/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:10.012332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.022501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e52f9-0dcb-4815-bb07-94f14403db38 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.035935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:10.084574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.085090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.085421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.085717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.086350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.088093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafb85e-7507-4e12-a761-b26dffbbc63e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.098600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1181/2401] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:10.103397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.113545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0270e10-b540-4102-83a2-a7c0c1d46afc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.119130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.119815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.120094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.120371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] PATH_INFO: `/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8/users/d61fd993f4a643a49d773d1a7cea11fa/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.122346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b94c53bb2b34b84b2dddf374044f2e8, user_id=d61fd993f4a643a49d773d1a7cea11fa, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.125086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.135549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1221/2402] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:10 2022] PUT /identity/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8/users/d61fd993f4a643a49d773d1a7cea11fa/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:10.146713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559215-f2d9-4c69-8281-7699161803de None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.147003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559215-f2d9-4c69-8281-7699161803de None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.147283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53559215-f2d9-4c69-8281-7699161803de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.175953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-53559215-f2d9-4c69-8281-7699161803de None None] MFA Rules not processed for user `d61fd993f4a643a49d773d1a7cea11fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:10.177162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-53559215-f2d9-4c69-8281-7699161803de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.244143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1182/2403] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:10.255214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.256981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.314342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a74e4-463f-41ab-aff0-347712470cbe admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.318013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.318598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.318893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.319168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.327937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.329355 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc080b0f-7a32-414c-93dc-528fabcfabc9 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.339125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1222/2404] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:10.405600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53559215-f2d9-4c69-8281-7699161803de None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.467729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53559215-f2d9-4c69-8281-7699161803de None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.472317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.472949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.473226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.473498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] PATH_INFO: `/v3/users/932fca992ede41479e90827e95b4387f` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.474645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=932fca992ede41479e90827e95b4387f)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.477132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.517636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:10.532350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932fca992ede41479e90827e95b4387f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.533093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932fca992ede41479e90827e95b4387f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.542599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932fca992ede41479e90827e95b4387f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.557808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932fca992ede41479e90827e95b4387f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.565183 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932fca992ede41479e90827e95b4387f'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:10.571559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1183/2405] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:10 2022] DELETE /identity/v3/users/932fca992ede41479e90827e95b4387f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:10.576069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.576356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.576635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.596860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:10.598135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.613277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:10.667884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1223/2406] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:10.677164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:10.679019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:10.738968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408fda-a3f2-4c84-8234-cbd43505bf2a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:10.742838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:10.743573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:10.743919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:10.744266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:10.754154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:10.755699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:10.766277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1184/2407] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:11.034015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.035507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.049880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a0321-2d8f-4e7d-864a-9c7b13d6d6ac None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.055542 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.056185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.056465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.056738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] PATH_INFO: `/v3/projects/5a2e8c4ef81a43f18138efd02bf96832` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.057896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2e8c4ef81a43f18138efd02bf96832)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.059956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.106836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2e8c4ef81a43f18138efd02bf96832', 'request_id': 'req-f30ffa58-d27a-4b34-b77e-d1a53c207d52'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.112373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2e8c4ef81a43f18138efd02bf96832', 'request_id': 'req-f30ffa58-d27a-4b34-b77e-d1a53c207d52'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.119869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2e8c4ef81a43f18138efd02bf96832', 'request_id': 'req-f30ffa58-d27a-4b34-b77e-d1a53c207d52'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.130644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] The token cache is being invalidate because project 5a2e8c4ef81a43f18138efd02bf96832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:11.130926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.131673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.133144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1224/2408] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:11 2022] DELETE /identity/v3/projects/5a2e8c4ef81a43f18138efd02bf96832 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:11.137119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.146765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986da1ab-1fb4-4dd3-b210-d88eb6cf5cbf service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.209373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.210545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.210825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.211095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] PATH_INFO: `/v3/users/d61fd993f4a643a49d773d1a7cea11fa` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.212274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61fd993f4a643a49d773d1a7cea11fa)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.214291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.261877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61fd993f4a643a49d773d1a7cea11fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.262517 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61fd993f4a643a49d773d1a7cea11fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.269737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61fd993f4a643a49d773d1a7cea11fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.275038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:11.283394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61fd993f4a643a49d773d1a7cea11fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.290439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1507ef69-3f73-42bc-8ad4-66b4293c94b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61fd993f4a643a49d773d1a7cea11fa'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.296250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1185/2409] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:11 2022] DELETE /identity/v3/users/d61fd993f4a643a49d773d1a7cea11fa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:11.575669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.577134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.590032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ffa58-d27a-4b34-b77e-d1a53c207d52 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.647290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.647898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.648178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.648447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] PATH_INFO: `/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.649557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b94c53bb2b34b84b2dddf374044f2e8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.651467 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.694010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:11.699604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b94c53bb2b34b84b2dddf374044f2e8', 'request_id': 'req-8400c8f6-f299-478e-8595-4c8478b9e9f2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.706214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b94c53bb2b34b84b2dddf374044f2e8', 'request_id': 'req-8400c8f6-f299-478e-8595-4c8478b9e9f2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.714587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b94c53bb2b34b84b2dddf374044f2e8', 'request_id': 'req-8400c8f6-f299-478e-8595-4c8478b9e9f2'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.728033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] The token cache is being invalidate because project 6b94c53bb2b34b84b2dddf374044f2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:11.728316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.728951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-8400c8f6-f299-478e-8595-4c8478b9e9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:11.730419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1225/2410] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:11 2022] DELETE /identity/v3/projects/6b94c53bb2b34b84b2dddf374044f2e8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:11.735403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.735712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.735983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.755975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:11.757213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.821580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1186/2411] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:11.830465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.830954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.831240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.852380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:11.853582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:11.862525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1226/2412] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:11.867973 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.885055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382f62e-abbd-421c-9e50-955eda915fa3 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.890081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.890689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.890970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.891235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.891943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:11.892246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.893858 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.906532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1187/2413] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:11.911181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.923141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67470d5a-ac97-48bc-91b1-ff802c727327 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:11.928311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:11.929029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:11.929385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:11.929724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:11.930435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:11.932477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:11.963996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1227/2414] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:11.968480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:11.979318 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bf771-359d-4dcf-8458-eaf11e2193fc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.046084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.046697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.046971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.047241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.047898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.049886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.070171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1188/2415] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:11 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:12.074103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.085763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8bcb-891f-426b-afd6-32e9c93e7183 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.090581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.091435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.091749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.092185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.092831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.094438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.105184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1228/2416] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.109262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.119422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73231db-0051-4673-b2fd-762fbd514444 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.124063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.124654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.124934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.125205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.125861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.127450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ab197-82c2-424e-b933-9db2f7b2b71f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.137891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1189/2417] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.141942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.152417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad443438-9e62-43f9-a611-1118b375c7fc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.157120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.157705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.157981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.158252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] PATH_INFO: `/v3/projects/35ebcb871e85415db10cc05eda787a21/users/547cd2af53774e31bb06c7932d09cfc8/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.173212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ebcb871e85415db10cc05eda787a21, user_id=547cd2af53774e31bb06c7932d09cfc8, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.175975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.188480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1229/2418] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:12 2022] PUT /identity/v3/projects/35ebcb871e85415db10cc05eda787a21/users/547cd2af53774e31bb06c7932d09cfc8/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:12.200276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.200589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.200861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.230147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] MFA Rules not processed for user `547cd2af53774e31bb06c7932d09cfc8`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:12.231431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.266087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.300286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1190/2419] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:12.304826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.367763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841ed9-a083-40b9-bed5-f7114fe6d416 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.372568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.373165 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.373440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.373711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.374341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.376401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.403606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1230/2420] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:12.407841 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.418646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c507-1272-411a-a554-f0e7cc695712 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.475898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.476514 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.476824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.477096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.477737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.480040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.504900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1191/2421] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:12.507824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.519374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777b2f-b277-4f3f-a412-e466fea3ae2c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.537097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.538032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.538314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.538588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.539247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.540954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.556125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1231/2422] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.562698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.575115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c65b2-6ace-48c5-953d-38371cfd2f39 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.588657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.589360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.589616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.590029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] PATH_INFO: `/v3/projects/276e053475a84b5d8f21867254fb75b3/users/08eda4050bb746429edf6b3e262fe11a/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.612913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=276e053475a84b5d8f21867254fb75b3, user_id=08eda4050bb746429edf6b3e262fe11a, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.616214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.632335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1192/2423] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:12 2022] PUT /identity/v3/projects/276e053475a84b5d8f21867254fb75b3/users/08eda4050bb746429edf6b3e262fe11a/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:12.637352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.648775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8bfc-1c00-41b5-a862-a911912b4e74 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.699919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.726064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.726705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.726983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.727271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.727999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.729819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5534a-bb07-430c-9ece-01b2446ee084 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.741902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1232/2424] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:12.746384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.755205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.755814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.756284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.761818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15223ab2-d889-4184-8f0d-e8ce7642ab3b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.767467 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.768174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.768458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.768733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] PATH_INFO: `/v3/projects/276e053475a84b5d8f21867254fb75b3/users/08eda4050bb746429edf6b3e262fe11a/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.770892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=276e053475a84b5d8f21867254fb75b3, user_id=08eda4050bb746429edf6b3e262fe11a, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:12.773758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba03846-ad29-48ab-8b1a-b57f2ed7df93 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:12.778169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:12.779664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.787080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1193/2425] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:12 2022] PUT /identity/v3/projects/276e053475a84b5d8f21867254fb75b3/users/08eda4050bb746429edf6b3e262fe11a/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:12.797911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.798201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.798594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:12.832592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] MFA Rules not processed for user `08eda4050bb746429edf6b3e262fe11a`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:12.833943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.843308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1233/2426] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:12.892753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.894258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.922846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1194/2427] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:12.936061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:12.938770 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:12.956189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:12.981383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f96dd2-0654-419a-90d2-f8cc8c32a938 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:12.987359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:12.988342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:12.988626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:12.988897 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.000346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.002035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.011842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1234/2428] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:50:13.013727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92e16-fd36-476e-96b2-3d0e9b505e8e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.017340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.017947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.018310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.018583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.029407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.030980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.045092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1195/2429] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:13.743789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.746238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:13.903595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:13.941817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087c5ab-39a1-49b3-84a5-b054459bbe91 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:13.953838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:13.954727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:13.955184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:13.955701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] PATH_INFO: `/v3/users/3637a2b86e4d44e2beaf692307447bd8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:13.964133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3637a2b86e4d44e2beaf692307447bd8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:13.966614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:13.994063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:14.060188 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3637a2b86e4d44e2beaf692307447bd8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.061012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3637a2b86e4d44e2beaf692307447bd8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.080569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3637a2b86e4d44e2beaf692307447bd8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.089311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a1a1-1db8-4337-ae7d-489cf4314002 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.103286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.104133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.104663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.104815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3637a2b86e4d44e2beaf692307447bd8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.105144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] PATH_INFO: `/v3/users/547cd2af53774e31bb06c7932d09cfc8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.108144 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=547cd2af53774e31bb06c7932d09cfc8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.113635 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.140495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2e6d832f-17c7-4d00-b9d9-537a2aa60f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3637a2b86e4d44e2beaf692307447bd8'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.153335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1235/2430] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/users/3637a2b86e4d44e2beaf692307447bd8 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:14.161239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.161553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.161798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.219496 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:14.221673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.236041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547cd2af53774e31bb06c7932d09cfc8'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.236914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547cd2af53774e31bb06c7932d09cfc8'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.262063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547cd2af53774e31bb06c7932d09cfc8'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.298053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547cd2af53774e31bb06c7932d09cfc8'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.316376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-cd0712a6-3acf-462b-86c1-7febcab3fe7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547cd2af53774e31bb06c7932d09cfc8'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.331368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1196/2431] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:13 2022] DELETE /identity/v3/users/547cd2af53774e31bb06c7932d09cfc8 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:14.332959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.333252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.333524 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.368122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:14.371057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.409554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1236/2432] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:14.426026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.429145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.444133 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:14.508217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1197/2433] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:14.527254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.528774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.546085 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0ae69-9250-4365-b7c0-22a882c658f1 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.550101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6ccb3-ed0f-4784-aac5-59afedcf6c10 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.560793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.561398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.561667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.561936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] PATH_INFO: `/v3/users/221d5130322d4d99bcf1fda9e2c79856` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.563102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221d5130322d4d99bcf1fda9e2c79856)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.568753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.611434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:14.625309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.626596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.628131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.628707 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.645712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.648795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.666029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221d5130322d4d99bcf1fda9e2c79856'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.667229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221d5130322d4d99bcf1fda9e2c79856'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.673176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1237/2434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:14.674286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.676723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221d5130322d4d99bcf1fda9e2c79856'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.694497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221d5130322d4d99bcf1fda9e2c79856'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.700019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8607-849b-4a4b-b35e-bf9a322fe4a0 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:14.703453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3348b62-18f2-4541-b2ef-b35a53d6e286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221d5130322d4d99bcf1fda9e2c79856'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:14.704606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:14.705219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.705644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.705923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.712438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.714280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1198/2435] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:14 2022] DELETE /identity/v3/users/221d5130322d4d99bcf1fda9e2c79856 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:14.716069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:14.716451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:14.716824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:14.744748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:14.746239 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.759910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:14.761381 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:14.783841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1238/2436] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:14.862438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1199/2437] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:14.872980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:14.874915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:14.888376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cef36d-9f33-4e46-920a-ef4d041e29d2 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.036765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.038568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.038962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.039274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.051852 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.053856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.066321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1239/2438] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:15.376728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:15.379658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.417604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c573-a28c-45d5-8d51-9757199eff18 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.433812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.434691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.435027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.435402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] PATH_INFO: `/v3/projects/35ebcb871e85415db10cc05eda787a21` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.436740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ebcb871e85415db10cc05eda787a21)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.439888 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.453282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:15.456046 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.485170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347d1d5-d568-4371-a313-2ed4bab9a0ae service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.497632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.498742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.499105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.500079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] PATH_INFO: `/v3/projects/c88aa1104bc6412a9b604de7db1bbdb8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.501481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88aa1104bc6412a9b604de7db1bbdb8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.504010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.564267 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:15.574118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88aa1104bc6412a9b604de7db1bbdb8', 'request_id': 'req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.581112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88aa1104bc6412a9b604de7db1bbdb8', 'request_id': 'req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.610136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ebcb871e85415db10cc05eda787a21', 'request_id': 'req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.619514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88aa1104bc6412a9b604de7db1bbdb8', 'request_id': 'req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.626572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ebcb871e85415db10cc05eda787a21', 'request_id': 'req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.640110 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ebcb871e85415db10cc05eda787a21', 'request_id': 'req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.644150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] The token cache is being invalidate because project c88aa1104bc6412a9b604de7db1bbdb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:15.645265 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.646119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b5d06a75-81fe-4c3f-aa1b-1dc8b19f4013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.650226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1240/2439] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:15 2022] DELETE /identity/v3/projects/c88aa1104bc6412a9b604de7db1bbdb8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:15.658071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.658071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.658071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.669388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] The token cache is being invalidate because project 35ebcb871e85415db10cc05eda787a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:15.669620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.670773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:15.678684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1200/2440] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:15 2022] DELETE /identity/v3/projects/35ebcb871e85415db10cc05eda787a21 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:15.678833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:15.688474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.702211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:15.706133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.802790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014e3c-255b-4b0c-b1ce-8186faf2f9e3 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.869839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1241/2441] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:15.876994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:15.880704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:15.887926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.888893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.889181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.889456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] PATH_INFO: `/v3/users/08eda4050bb746429edf6b3e262fe11a` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.891083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08eda4050bb746429edf6b3e262fe11a)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.894294 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:15.895737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:15.904621 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d957f0-5f08-4317-a666-d32c7dbed3ff None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:15.920533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:15.921390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:15.921680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:15.922001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] PATH_INFO: `/v3/projects/d836437226dc4eb485ba02cc264fb072` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:15.925475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d836437226dc4eb485ba02cc264fb072)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:15.927440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:16.030940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08eda4050bb746429edf6b3e262fe11a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.034138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08eda4050bb746429edf6b3e262fe11a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.059812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08eda4050bb746429edf6b3e262fe11a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.097867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08eda4050bb746429edf6b3e262fe11a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.120704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:16.128299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-c3f3ec55-1b3f-4eb7-a9ea-567133b82d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08eda4050bb746429edf6b3e262fe11a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.132873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd836437226dc4eb485ba02cc264fb072', 'request_id': 'req-5770753d-9117-4f96-9281-e3c5affbf961'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.139402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1201/2442] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:15 2022] DELETE /identity/v3/users/08eda4050bb746429edf6b3e262fe11a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:16.146205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:16.147129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:16.147412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:16.147901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd836437226dc4eb485ba02cc264fb072', 'request_id': 'req-5770753d-9117-4f96-9281-e3c5affbf961'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.173792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd836437226dc4eb485ba02cc264fb072', 'request_id': 'req-5770753d-9117-4f96-9281-e3c5affbf961'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.193052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] The token cache is being invalidate because project d836437226dc4eb485ba02cc264fb072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:16.203875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.203875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.203875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1242/2443] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:15 2022] DELETE /identity/v3/projects/d836437226dc4eb485ba02cc264fb072 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:16.203875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:16.218872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:16.239534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:16.241628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:16.394196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770753d-9117-4f96-9281-e3c5affbf961 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:16.395932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1202/2444] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:16.400790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:16.420681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:16.421948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:16.423363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:16.424747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] PATH_INFO: `/v3/users/f7e4a23f1d9a493792cbc04887b7c0c7` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:16.427654 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e4a23f1d9a493792cbc04887b7c0c7)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:16.435691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:16.440762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b64b4c-aa40-418e-aa36-e57bf3136b1d None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:16.478274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:16.478599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:16.478867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:16.479233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:16.482233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:16.482566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:16.484341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:16.552018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1203/2445] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:16.555508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:16.565185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e4a23f1d9a493792cbc04887b7c0c7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.565941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e4a23f1d9a493792cbc04887b7c0c7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.581551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbe68a-bc40-4ac6-859f-d282c7412641 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:16.588915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e4a23f1d9a493792cbc04887b7c0c7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.603537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e4a23f1d9a493792cbc04887b7c0c7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.605985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:16.606609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:16.606885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:16.607152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:16.616782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:16.622588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:16.656103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e4a23f1d9a493792cbc04887b7c0c7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:16.687599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1243/2446] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:16 2022] DELETE /identity/v3/users/f7e4a23f1d9a493792cbc04887b7c0c7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:16.732378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1204/2447] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:16 2022] POST /identity/v3/projects => generated 397 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:16.737119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:16.738901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:16.754589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:16.887992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f92b80-efde-483e-8249-baa071dd6f9c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.022771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.023316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.023697 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.024007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.025191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.027550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.079021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.104581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1244/2448] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:16 2022] POST /identity/v3/users => generated 299 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:17.119980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b3ab-f437-4884-b2bd-93919febb43b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.129802 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.131598 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.139503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.139972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.140229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.141783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.142144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.147473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.165908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cf967-6651-4387-9ee8-186891114edc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.177415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.178110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.178394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.178665 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] PATH_INFO: `/v3/projects/276e053475a84b5d8f21867254fb75b3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.181903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=276e053475a84b5d8f21867254fb75b3)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.184267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.191293 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1205/2449] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:17.194209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.238680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2108b-06c2-4166-8c58-7f3cfd77bace admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.265237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276e053475a84b5d8f21867254fb75b3', 'request_id': 'req-10f05672-6c40-4535-a7f1-ef6a387b2a34'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.280462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276e053475a84b5d8f21867254fb75b3', 'request_id': 'req-10f05672-6c40-4535-a7f1-ef6a387b2a34'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.299521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276e053475a84b5d8f21867254fb75b3', 'request_id': 'req-10f05672-6c40-4535-a7f1-ef6a387b2a34'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.329802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] The token cache is being invalidate because project 276e053475a84b5d8f21867254fb75b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:17.329994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.335546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-10f05672-6c40-4535-a7f1-ef6a387b2a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.345764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1245/2450] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:17 2022] DELETE /identity/v3/projects/276e053475a84b5d8f21867254fb75b3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:17.347698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf259f-f128-4a90-becd-9d9bac37e4db None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.347984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf259f-f128-4a90-becd-9d9bac37e4db None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.348256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf259f-f128-4a90-becd-9d9bac37e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.377828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.378448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.378727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.379608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.379742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.383534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.397733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-7baf259f-f128-4a90-becd-9d9bac37e4db None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:17.398893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7baf259f-f128-4a90-becd-9d9bac37e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.403725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1206/2451] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:17.409862 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.414280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.429739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1246/2452] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:17.437748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.438100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.438299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.464305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040dcc5-792c-4bbb-bebd-a2a45d164991 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.472960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.473885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.474330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.474803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] PATH_INFO: `/v3/projects/093250be47d3423681e74ebea66b8b81/users/8224d49a515a4b878752afe168f9f72d/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.492730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:17.493870 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.514926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=093250be47d3423681e74ebea66b8b81, user_id=8224d49a515a4b878752afe168f9f72d, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.519181 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1247/2453] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:17.524103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.526297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487640b-2d48-4b1e-b4f8-429c6eed5a1a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.534271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77d4af-ff74-476c-a778-10e45021a19a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.548928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.549736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.550020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.550295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.551562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:17.551942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.555586 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.562686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1207/2454] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:17 2022] PUT /identity/v3/projects/093250be47d3423681e74ebea66b8b81/users/8224d49a515a4b878752afe168f9f72d/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:17.577246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352576-a262-40ef-a38c-c93645cba788 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.577678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352576-a262-40ef-a38c-c93645cba788 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.578035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42352576-a262-40ef-a38c-c93645cba788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.580806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:17.594493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1248/2455] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:17.596338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.599432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.616207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d247e0-315a-4313-b462-bcdf78b5646e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.629134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-42352576-a262-40ef-a38c-c93645cba788 None None] MFA Rules not processed for user `8224d49a515a4b878752afe168f9f72d`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:17.630529 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-42352576-a262-40ef-a38c-c93645cba788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.642855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:17.693053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.694481 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.695099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.695500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] PATH_INFO: `/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.696759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3243693955cc4d2fa97c70ee4bc9f56e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.698916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.713683 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1208/2456] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:17.714871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42352576-a262-40ef-a38c-c93645cba788 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.727177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42352576-a262-40ef-a38c-c93645cba788 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.732870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.734662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.734662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.734662 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.735238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:17.737314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:17.774727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1209/2457] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:17 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:17.775864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.777527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.785648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3243693955cc4d2fa97c70ee4bc9f56e', 'request_id': 'req-b9fa6bf8-78df-4138-aaa0-973cf5599762'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.801072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3243693955cc4d2fa97c70ee4bc9f56e', 'request_id': 'req-b9fa6bf8-78df-4138-aaa0-973cf5599762'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.867708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3243693955cc4d2fa97c70ee4bc9f56e', 'request_id': 'req-b9fa6bf8-78df-4138-aaa0-973cf5599762'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.890070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] The token cache is being invalidate because project 3243693955cc4d2fa97c70ee4bc9f56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:17.890794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.891611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:17.898962 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1249/2458] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:17 2022] DELETE /identity/v3/projects/3243693955cc4d2fa97c70ee4bc9f56e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:17.903742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:17.908194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:17.968393 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3de8a-b31e-4450-9368-4aece6f210a7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:17.985825 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:17.986474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:17.986755 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:17.987055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:17.994584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:18.124896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6bf8-78df-4138-aaa0-973cf5599762 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.154903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.157230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.168931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1210/2459] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:18.175001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:18.218391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.219031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.219460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.220394 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.221332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.224035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.266392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1250/2460] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:17 2022] POST /identity/v3/users => generated 292 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:18.267769 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:18.279300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4f8bf-2bfe-4151-88f5-3f33bec29901 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.285394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.285780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.286061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.286332 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.286975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.290828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.307920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1211/2461] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:18.309333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:18.320641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92381fd-d9d9-4515-a5f8-c9eea4470551 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.326433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.327126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.327519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.328034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.328492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.330153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0ad6d-dc8e-4274-b937-d4154d2f0e59 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.350194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1251/2462] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:18.352743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:18.364238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca62a7-c169-407d-89fb-6d558427d2e4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.386383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.387272 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.387597 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.387875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] PATH_INFO: `/v3/projects/af388a710ff046809741d209147c75e9/users/76e71075e556481ab968915916f8405b/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.413968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af388a710ff046809741d209147c75e9, user_id=76e71075e556481ab968915916f8405b, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.419777 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.440692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1212/2463] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:18 2022] PUT /identity/v3/projects/af388a710ff046809741d209147c75e9/users/76e71075e556481ab968915916f8405b/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:18.452722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.453018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.453411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.506795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] MFA Rules not processed for user `76e71075e556481ab968915916f8405b`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:18.509444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:18.622295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1252/2464] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:18.627029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:18.652457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:18.789987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a99102-ae8e-444f-ac28-1735f3757fd4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.804127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.805223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.805223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.805428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.805941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.808147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:18.914722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:18.925822 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1213/2465] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:18 2022] POST /identity/v3/projects => generated 379 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:18.926942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fce31-4e86-4b0b-9ba7-f3f14d174c27 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:18.995284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:18.995940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:18.996221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:18.996495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:18.997125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:18.999065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.028061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1253/2466] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:18 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:19.031830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.043682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d31bc7-a670-4be4-8204-fa65857581f9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.048474 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.049084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.049359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.049750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.050749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.052462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.067175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1214/2467] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:19.068921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.079809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e456085-c727-4f35-8b28-9e02b00a3de4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.085562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.086163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.086440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.086711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.087349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.089018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe4999-2e30-40e3-adfe-27e6fd9738fd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.099778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1254/2468] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:19.104238 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.116713 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23b6d-21c4-426b-8b4d-84b32cc76dad admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.121513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.122200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.122483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.122846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] PATH_INFO: `/v3/projects/8e5183a884484022b648c4df0e55dde9/users/5c6011c2eb7e496faeeb490a0283e361/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.142091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e5183a884484022b648c4df0e55dde9, user_id=5c6011c2eb7e496faeeb490a0283e361, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.144906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.162200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1215/2469] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:19 2022] PUT /identity/v3/projects/8e5183a884484022b648c4df0e55dde9/users/5c6011c2eb7e496faeeb490a0283e361/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:19.173525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.173986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.174366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.207290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] MFA Rules not processed for user `5c6011c2eb7e496faeeb490a0283e361`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:19.208740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:19.222909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:19.291403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1255/2470] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:19.292516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.396648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0540fbb-a47c-4726-adec-39d6ef82e8cf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.409276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.412346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.412905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.413396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.414124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.421300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.471512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1216/2471] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:19 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:50:19.475550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.486753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3add37-36b1-4b95-b029-e2de483005f6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.555356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.556415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.556807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.557318 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.558233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.560207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.587583 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1256/2472] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:19 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:19.591070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.603404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308d05-fa43-4194-a23b-3cdd53f76bc1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.608593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.609345 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.609733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.610359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.611035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.612998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.626972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1217/2473] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:19.632204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.643403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41698e-265a-499d-bacd-b5511cdceaac admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.648549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.649362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.649648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.649923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] PATH_INFO: `/v3/projects/0164d0b3fc8f41b586aea649ec10bba6/users/8dc0d5e10ada4ef099615aafa41a8289/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.667432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0164d0b3fc8f41b586aea649ec10bba6, user_id=8dc0d5e10ada4ef099615aafa41a8289, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.670211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.686880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1257/2474] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:19 2022] PUT /identity/v3/projects/0164d0b3fc8f41b586aea649ec10bba6/users/8dc0d5e10ada4ef099615aafa41a8289/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:19.690933 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.710940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebefbe-5824-48a2-94d8-cec97cb4ebca admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.746968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:19.779190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.779838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.780120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.780391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.781056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.782711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602381b-dc91-491e-99e6-04a1bea3a948 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.794502 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1218/2475] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:19.798402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:19.810254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eed3a-f6ab-4465-9678-381943cd1ee2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:19.817475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:19.818404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.818764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.819102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] PATH_INFO: `/v3/projects/0164d0b3fc8f41b586aea649ec10bba6/users/8dc0d5e10ada4ef099615aafa41a8289/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.821167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0164d0b3fc8f41b586aea649ec10bba6, user_id=8dc0d5e10ada4ef099615aafa41a8289, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:19.823936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:19.837443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1258/2476] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:19 2022] PUT /identity/v3/projects/0164d0b3fc8f41b586aea649ec10bba6/users/8dc0d5e10ada4ef099615aafa41a8289/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:19.848099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:19.848389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:19.848666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:19.886150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] MFA Rules not processed for user `8dc0d5e10ada4ef099615aafa41a8289`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:19.887720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:20.071577 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1219/2477] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:20.090006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:20.092727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:20.232724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f463b-4e11-4c0d-9b4f-9ee9c7d95215 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:20.237516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:20.242214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:20.242619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:20.242993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:20.265658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:20.332018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:20.333711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:20.353122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1259/2478] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:25.254696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:25.267109 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53661ff-bc7b-4c12-b713-a74c31e19214 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:25.270944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:25.271814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:25.272190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:25.272552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:25.335396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:25.336885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:25.348271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1220/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:25.989196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:26.007006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831361a-8be4-4249-8d3f-203ab503e733 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:26.032388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:26.032917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:26.033203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:26.033478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:26.055017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:26.056933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20326ec3-2c27-45ea-8f75-ed743f3f1661 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:26.076796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1260/2480] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:27.490646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:27.492452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:27.553923 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:27.571682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648e603-1178-48dc-96c1-a727618e3f80 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:27.577558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:27.578306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.578658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.579010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] PATH_INFO: `/v3/users/8224d49a515a4b878752afe168f9f72d` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.580402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8224d49a515a4b878752afe168f9f72d)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:27.582955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:27.678859 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8224d49a515a4b878752afe168f9f72d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:27.679599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8224d49a515a4b878752afe168f9f72d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:27.706453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8224d49a515a4b878752afe168f9f72d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:27.734667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8224d49a515a4b878752afe168f9f72d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:27.752763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-84b151ed-4c4a-4da4-a46b-38ab9c51c36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8224d49a515a4b878752afe168f9f72d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:27.772592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.773118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.773370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.776076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1221/2481] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:27 2022] DELETE /identity/v3/users/8224d49a515a4b878752afe168f9f72d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:27.807723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:27.807723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:27.905389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1261/2482] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:27.905628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.905990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.906342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.915864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:27.917290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:27.928649 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] MFA Rules not processed for user `7d3139d9e58f4b56a370661e2a5d4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:27.929327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:27.974924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68d210-4dfc-49aa-ae80-1c4216315b70 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:27.978499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:27.979102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:27.979372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:27.979672 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:27.988191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1222/2483] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:27.993286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:27.994855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c026e-8126-4ed4-9620-2a3ec49f5c21 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.018354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1262/2484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:28.328782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.330233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:28.345067 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abac555-8afd-4df9-8d36-373d1c357280 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.363059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.366498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.366801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.367180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] PATH_INFO: `/v3/projects/093250be47d3423681e74ebea66b8b81` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.369089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093250be47d3423681e74ebea66b8b81)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.371206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.425801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093250be47d3423681e74ebea66b8b81', 'request_id': 'req-3c107c2f-0b85-416d-b088-31ccbc74e252'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:28.432192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093250be47d3423681e74ebea66b8b81', 'request_id': 'req-3c107c2f-0b85-416d-b088-31ccbc74e252'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:28.440340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093250be47d3423681e74ebea66b8b81', 'request_id': 'req-3c107c2f-0b85-416d-b088-31ccbc74e252'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:28.449657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:28.454663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] The token cache is being invalidate because project 093250be47d3423681e74ebea66b8b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:28.455646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:28.456361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-3c107c2f-0b85-416d-b088-31ccbc74e252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:28.458184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1223/2485] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:28 2022] DELETE /identity/v3/projects/093250be47d3423681e74ebea66b8b81 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:28.463936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.464261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.464539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.485702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:28.487131 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:28.550999 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:28.567068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1263/2486] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:28.575457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.576007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.576007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.596233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:28.597538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:28.605903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1224/2487] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:28.610193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.621198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1cdcb-c32a-41fd-8781-bfa4bc6a4264 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.625816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.626416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.626694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.626963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.628293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:28.628595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.630200 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.643515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1264/2488] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:28.648154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.657580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b81c7-6bf8-4572-be48-8602a946aa2e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.662823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.663453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.663765 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.664052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.664714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.666629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.696364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1225/2489] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:28 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:28.700812 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.713595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb2e10-081f-4f31-8f67-641b6f7ec19b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.785084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.785727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.786072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.786450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.787722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.789973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.819150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1265/2490] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:28 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:28.823456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.839289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a357de-f1cb-4618-90dd-32ebd1d93fa9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.848041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.850916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.851310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.851956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.853259 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.856137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.877493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1226/2491] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:28.878691 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.903532 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a710180-7236-4a32-8227-47208edb0316 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.921215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.922114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.922493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.922835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:28.923775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:28.925860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09779a-bd60-4d7a-871d-59e791b7fd11 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:28.943057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1266/2492] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:28.946926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:28.963844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c5097-87f8-45ba-8e5c-438543b5b1db admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:28.971832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:28.973056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:28.973301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:28.973797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] PATH_INFO: `/v3/projects/79ef0ce1ee884e9d80d4200d40b7b6c2/users/c75bb50ad7704e66983959d338b48e27/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:29.003030 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ef0ce1ee884e9d80d4200d40b7b6c2, user_id=c75bb50ad7704e66983959d338b48e27, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:29.006712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:29.027243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1227/2493] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:28 2022] PUT /identity/v3/projects/79ef0ce1ee884e9d80d4200d40b7b6c2/users/c75bb50ad7704e66983959d338b48e27/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:50:29.032325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:29.033067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:29.033201 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:29.069103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:29.070816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:29.168745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1267/2494] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:29.180019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:29.182301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:29.256615 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4edc8c-4af1-43dc-98cc-be7d48a155cb admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:29.263097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:29.264091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:29.264396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:29.264672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:29.277566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:29.279299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7cb8f-be13-4a27-8562-5fbf2834e942 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:29.290530 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1228/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:30.457705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:30.472782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:30.492944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64194ef-8c54-4580-b7e2-6eaafa3ea645 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:30.509712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:30.510477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:30.510825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:30.511437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] PATH_INFO: `/v3/users/76e71075e556481ab968915916f8405b` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:30.512702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e71075e556481ab968915916f8405b)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:30.514916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:30.675422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e71075e556481ab968915916f8405b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:30.676072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e71075e556481ab968915916f8405b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:30.684506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:30.699909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e71075e556481ab968915916f8405b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:30.718721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e71075e556481ab968915916f8405b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:30.749591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e71075e556481ab968915916f8405b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:30.762538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1268/2496] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:30 2022] DELETE /identity/v3/users/76e71075e556481ab968915916f8405b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:30.764603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:30.765154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:30.765386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:30.794438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:30.796297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:30.903521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1229/2497] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:30.915631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:30.917527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:31.046057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588745a7-1a57-4f9f-89aa-5221d9bea8ad admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:31.053886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:31.054721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:31.055099 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:31.055447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:31.078750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:31.080275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:31.101995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1269/2498] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:50:32.089843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:32.092202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:32.108840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcc8e2-a82d-46cb-910e-ffff903690f9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:32.117746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:32.118607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:32.119060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:32.119722 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] PATH_INFO: `/v3/projects/af388a710ff046809741d209147c75e9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:32.120895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af388a710ff046809741d209147c75e9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:32.122843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:32.179544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:32.221327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af388a710ff046809741d209147c75e9', 'request_id': 'req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.240046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af388a710ff046809741d209147c75e9', 'request_id': 'req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.254526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af388a710ff046809741d209147c75e9', 'request_id': 'req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.270644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] The token cache is being invalidate because project af388a710ff046809741d209147c75e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:32.271674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.272708 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.278906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:32.279657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1230/2499] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:32 2022] DELETE /identity/v3/projects/af388a710ff046809741d209147c75e9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:32.291502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33bc7-3947-4433-b6cb-0cdf4ce232a4 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:32.384024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:32.385301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:32.385750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:32.386056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] PATH_INFO: `/v3/users/5c6011c2eb7e496faeeb490a0283e361` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:32.387337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c6011c2eb7e496faeeb490a0283e361)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:32.389398 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:32.486807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c6011c2eb7e496faeeb490a0283e361'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.488098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c6011c2eb7e496faeeb490a0283e361'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.504093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c6011c2eb7e496faeeb490a0283e361'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.523004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c6011c2eb7e496faeeb490a0283e361'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.538244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:32.548502 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c6011c2eb7e496faeeb490a0283e361'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:32.560058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1270/2500] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:32 2022] DELETE /identity/v3/users/5c6011c2eb7e496faeeb490a0283e361 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:33.167226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:33.171582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:33.209607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bcfc4-6ab0-4e05-8e8b-e3d439b2305e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:33.368376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:33.368376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:33.368376 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:33.369806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] PATH_INFO: `/v3/projects/8e5183a884484022b648c4df0e55dde9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:33.373335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e5183a884484022b648c4df0e55dde9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:33.377986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:33.440173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e5183a884484022b648c4df0e55dde9', 'request_id': 'req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.446975 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e5183a884484022b648c4df0e55dde9', 'request_id': 'req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.458616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e5183a884484022b648c4df0e55dde9', 'request_id': 'req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.476390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] The token cache is being invalidate because project 8e5183a884484022b648c4df0e55dde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:33.477609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.478463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.483751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1231/2501] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:33 2022] DELETE /identity/v3/projects/8e5183a884484022b648c4df0e55dde9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:33.484454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:33.486559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:33.508994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd4d2c-80a5-4494-90dc-c2398efbcfb3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:33.651140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:33.651833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:33.652114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:33.652390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] PATH_INFO: `/v3/users/8dc0d5e10ada4ef099615aafa41a8289` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:33.653590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc0d5e10ada4ef099615aafa41a8289)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:33.655568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:33.780295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc0d5e10ada4ef099615aafa41a8289'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.781313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc0d5e10ada4ef099615aafa41a8289'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.807849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc0d5e10ada4ef099615aafa41a8289'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.820984 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc0d5e10ada4ef099615aafa41a8289'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.862340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-f5af64af-b3f1-4655-82db-687c6bce88e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc0d5e10ada4ef099615aafa41a8289'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:33.880779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1271/2502] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:33 2022] DELETE /identity/v3/users/8dc0d5e10ada4ef099615aafa41a8289 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:34.608956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:34.611010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:34.626070 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335bdf3-ec40-4c33-8ef1-ed79e0fb1dcf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:34.687725 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:34.701242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:34.701986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:34.705423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:34.705423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] PATH_INFO: `/v3/projects/0164d0b3fc8f41b586aea649ec10bba6` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:34.705423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0164d0b3fc8f41b586aea649ec10bba6)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:34.718018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:34.821122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0164d0b3fc8f41b586aea649ec10bba6', 'request_id': 'req-f805d467-889c-4a47-91d5-6de76015bc04'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.829783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0164d0b3fc8f41b586aea649ec10bba6', 'request_id': 'req-f805d467-889c-4a47-91d5-6de76015bc04'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.849466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0164d0b3fc8f41b586aea649ec10bba6', 'request_id': 'req-f805d467-889c-4a47-91d5-6de76015bc04'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.873353 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] The token cache is being invalidate because project 0164d0b3fc8f41b586aea649ec10bba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:34.873654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.876395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:34.896282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1232/2503] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:34 2022] DELETE /identity/v3/projects/0164d0b3fc8f41b586aea649ec10bba6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:35.788069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:35.788963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:35.789629 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:35.833893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] MFA Rules not processed for user `c75bb50ad7704e66983959d338b48e27`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:35.835126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:35.921126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:35.926580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1272/2504] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:35.931291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:35.932869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:36.039794 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805d467-889c-4a47-91d5-6de76015bc04 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.048116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.050139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.050435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.050853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.053741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.056942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.130068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1233/2505] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:35 2022] POST /identity/v3/projects => generated 373 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:36.134679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.149253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76d39b-204c-4f6d-81ff-4670805c1150 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.253643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.254434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.254814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.255301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.258978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.261552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.298053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1273/2506] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:36 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:36.302784 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.321118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d5f17-6da7-4eb6-965a-16e9de8930b7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.333902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.335240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.336039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.336609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.337584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.339641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.358542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1234/2507] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:36.362379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.375455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4823d-139f-46a4-8bf6-6f8a2e47dfca admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.380435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.381109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.381397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.381677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] PATH_INFO: `/v3/projects/3aabd6541e7e4d429574cedba7e6e607/users/3133926f799346b48699867982fb9a6b/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.403998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aabd6541e7e4d429574cedba7e6e607, user_id=3133926f799346b48699867982fb9a6b, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.406879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.424150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1274/2508] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:36 2022] PUT /identity/v3/projects/3aabd6541e7e4d429574cedba7e6e607/users/3133926f799346b48699867982fb9a6b/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:36.428834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.440840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ccf3-9fbb-4758-98a3-c4ee8e2c3eed admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.541355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:36.580235 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.581180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.581629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.582075 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.582877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.585306 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474977c-ee73-4c7c-a165-03458e81a537 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.602943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1235/2509] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:36.604197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:36.615928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc8e9-7e94-4b88-911b-521c4bb0ef6c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:36.621326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:36.622011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:36.622294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:36.622573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] PATH_INFO: `/v3/projects/3aabd6541e7e4d429574cedba7e6e607/users/3133926f799346b48699867982fb9a6b/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:36.624975 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aabd6541e7e4d429574cedba7e6e607, user_id=3133926f799346b48699867982fb9a6b, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:36.627722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:36.641519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1275/2510] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:36 2022] PUT /identity/v3/projects/3aabd6541e7e4d429574cedba7e6e607/users/3133926f799346b48699867982fb9a6b/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:41.125071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187a205-6162-4fd2-8bdf-bf54e9afabc2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:41.125757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187a205-6162-4fd2-8bdf-bf54e9afabc2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:41.125757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187a205-6162-4fd2-8bdf-bf54e9afabc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:41.182057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c187a205-6162-4fd2-8bdf-bf54e9afabc2 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:41.196277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c187a205-6162-4fd2-8bdf-bf54e9afabc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:41.386599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1236/2511] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:50:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:41.420335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:41.425271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:41.600649 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d919ef-ed32-426f-a35a-db8a942b3da1 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:41.616198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:41.617044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:41.617309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:41.617579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:41.656210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:41.657890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:41.682775 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1276/2512] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:50:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:50:44.498131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd463-0731-4fb9-83e9-047af7ed493d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.499046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd463-0731-4fb9-83e9-047af7ed493d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.499548 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bd463-0731-4fb9-83e9-047af7ed493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.538576 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a59bd463-0731-4fb9-83e9-047af7ed493d None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:44.541985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a59bd463-0731-4fb9-83e9-047af7ed493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:44.558441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1237/2513] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:50:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:44.573369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:44.586122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f85150-a9f2-4a99-961b-259b8274521b service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:44.604615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:44.605250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:44.605531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:44.605804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:44.627565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:44.629160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:44.650571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1277/2514] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:50:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:50:46.390336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.392023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.392525 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.550184 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] MFA Rules not processed for user `3133926f799346b48699867982fb9a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:46.553083 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.707696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1238/2515] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:46.726529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:46.728335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:46.799762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:46.852079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a826-3781-4b45-9e93-c29018a5edcb service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:46.858068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:46.858758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:46.859266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:46.859736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:46.971397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:46.973050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:46.987848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1278/2516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:47.549290 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:47.552275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:47.651939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468fc1-75b8-4c35-b3c0-e91f426378a2 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:47.658011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:47.658734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:47.659431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:47.659808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] PATH_INFO: `/v3/users/4aa761298e1f45e08fe4e42fba210d2b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:47.661511 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa761298e1f45e08fe4e42fba210d2b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:47.667217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:47.735684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:47.823816 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa761298e1f45e08fe4e42fba210d2b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.825946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa761298e1f45e08fe4e42fba210d2b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.853473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa761298e1f45e08fe4e42fba210d2b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.904778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa761298e1f45e08fe4e42fba210d2b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.919198 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-52dfee0f-0218-4ab0-802b-b2a082175852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa761298e1f45e08fe4e42fba210d2b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:47.930826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1239/2517] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:50:47 2022] DELETE /identity/v3/users/4aa761298e1f45e08fe4e42fba210d2b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:48.332081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.354866 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df6822-3cb0-436e-a671-64e3192ae685 service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.510503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.511772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.512407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.513040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] PATH_INFO: `/v3/projects/f62f652ab66a4289be10048e1c8169d6` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.515313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62f652ab66a4289be10048e1c8169d6)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.527136 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.675824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62f652ab66a4289be10048e1c8169d6', 'request_id': 'req-6067519a-39e6-4d57-aaa3-b25487240f68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.683602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62f652ab66a4289be10048e1c8169d6', 'request_id': 'req-6067519a-39e6-4d57-aaa3-b25487240f68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.705151 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62f652ab66a4289be10048e1c8169d6', 'request_id': 'req-6067519a-39e6-4d57-aaa3-b25487240f68'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.711995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:48.734952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] The token cache is being invalidate because project f62f652ab66a4289be10048e1c8169d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:50:48.737296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.738846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-6067519a-39e6-4d57-aaa3-b25487240f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:50:48.741371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1279/2518] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:50:48 2022] DELETE /identity/v3/projects/f62f652ab66a4289be10048e1c8169d6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:48.746092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.746609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.746860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.777914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:48.779689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.855837 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1240/2519] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:50:48.865835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.866360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.866876 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.891882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:48.893700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:48.912358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.928084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1280/2520] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:48.930856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0e854-4181-4736-98e1-c7543432d085 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.937591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.939652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.939652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.939652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.940123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:50:48.940647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:48.943383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:48.969180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:48.975459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1241/2521] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:50:48.982632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff776f-2a2a-4707-8a35-e3e0e6ffa580 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:48.990675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:48.992522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:48.994254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:48.994636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:48.995371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.000736 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.088504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.104527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1281/2522] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:49.111167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d2f02-5ceb-4d05-9a23-3c0c8aec8f0d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.236154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.237275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.237967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.238403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.239520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.243271 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.303526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.309240 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1242/2523] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:49.325395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a0df-c957-4b43-a5d0-ea3b5dfcf5bb admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.335217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.338187 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.338645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.338923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.339881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.341800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.366796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.380528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1282/2524] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:50:49.386158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7703b1d-4f10-4b30-9c4a-b57322c3ee2f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.396994 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.397886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.399385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.399920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.401685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.403944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff8a40-8253-402d-9eb5-21b3fe502386 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.425251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.443974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1243/2525] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:49.447700 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecb492-6422-45de-a30b-030cb33dad8e admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.458423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.459329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.459784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.460267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] PATH_INFO: `/v3/projects/b9cd3cd0a9434f5ab0e1e1c52145beb2/users/4279761de1464be38860432a46a89074/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.479759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9cd3cd0a9434f5ab0e1e1c52145beb2, user_id=4279761de1464be38860432a46a89074, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.484472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.520425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1283/2526] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:49 2022] PUT /identity/v3/projects/b9cd3cd0a9434f5ab0e1e1c52145beb2/users/4279761de1464be38860432a46a89074/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:49.526162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa985-eccb-475d-9581-47057dae7e36 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.526447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa985-eccb-475d-9581-47057dae7e36 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.526759 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa985-eccb-475d-9581-47057dae7e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.550381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:49.566192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d7eaa985-eccb-475d-9581-47057dae7e36 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:49.569746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d7eaa985-eccb-475d-9581-47057dae7e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.670103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1244/2527] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:49.680160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:49.682227 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:49.827389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68c4ce-1965-4e1a-8583-32b1a4e0cecd admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:49.835414 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:49.836130 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:49.836421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:49.836788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:49.847294 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:49.849323 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7f743-5382-4307-b157-ce00a147a5d5 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:49.862726 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1284/2528] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:50:50.064527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:50.065006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:50.065694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:50.094875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] MFA Rules not processed for user `c75bb50ad7704e66983959d338b48e27`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:50.097311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:50.200046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1245/2529] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:57.482976 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:57.483915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:57.484387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:57.556522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] MFA Rules not processed for user `4279761de1464be38860432a46a89074`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:50:57.558660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:50:57.732810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1285/2530] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:50:57.736992 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:57.758559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b793-d59b-4305-9b03-431b2a33d495 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:57.765806 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:57.766785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:57.767378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:57.767910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:57.768878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:57.771465 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:57.824172 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1246/2531] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:50:57 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:57.825118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:57.849486 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe16eb-4d00-4ce8-a170-5be1fc88c289 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:57.905980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:50:57.979599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:57.980391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:57.980796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:57.981159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:57.983523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:57.984039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.033562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1286/2532] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:50:57 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:50:58.034870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.053886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0cfed-2e6d-4c5a-8c12-6993a380f87e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.068856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.069786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.070648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.071099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.072153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.074867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.104313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1247/2533] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:58.115934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.141206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef360ce-ecec-410c-806b-9b6032dce8a4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.148230 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.148834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.149121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.149387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] PATH_INFO: `/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5/users/9d3e53bec1ed45a7a3a24cdd99c2dc01/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.209573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ab9fdc1b0422fbd49556f5c55f8a5, user_id=9d3e53bec1ed45a7a3a24cdd99c2dc01, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.213570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.295594 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1287/2534] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:58 2022] PUT /identity/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5/users/9d3e53bec1ed45a7a3a24cdd99c2dc01/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:50:58.297843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.313215 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ebd48-d29c-47ea-8f1c-965423629994 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.434503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.435291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.435701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.436292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.437276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.439130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a213cc-0fe4-4339-9e7b-5a98173d275f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.459471 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1248/2535] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:50:58.463730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:50:58.475679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8494d-3fcc-4e5a-a5ac-a522eabc4f13 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:50:58.481148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:50:58.481846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:50:58.482127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:50:58.482403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] PATH_INFO: `/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5/users/9d3e53bec1ed45a7a3a24cdd99c2dc01/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:50:58.484457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5ab9fdc1b0422fbd49556f5c55f8a5, user_id=9d3e53bec1ed45a7a3a24cdd99c2dc01, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:50:58.487144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:50:58.499835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1288/2536] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:50:58 2022] PUT /identity/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5/users/9d3e53bec1ed45a7a3a24cdd99c2dc01/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:51:05.284538 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:05.285603 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:05.286608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:05.323875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:05.344291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] MFA Rules not processed for user `9d3e53bec1ed45a7a3a24cdd99c2dc01`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:05.345461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.514205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1249/2537] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:51:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:05.524963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:05.526746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:05.629550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091545a-3d01-4c13-b76b-e4566071fcab admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:05.640362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:05.641549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:05.641908 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:05.642255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:05.726758 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:05.729049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c884cc-5474-4150-b70f-7d47a5693714 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:05.743405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1289/2538] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:09.984064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:09.985980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:10.070317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181309b-cca4-42e5-90a9-9a382d194862 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:10.075224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:10.075962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:10.076659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:10.076659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:10.078409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:10.093337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:10.097801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784286c-da9e-4899-ba99-11174f3359dc service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:10.112337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1250/2539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:12.323446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.324128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.324408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.325770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1290/2540] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:51:12.335991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d2969-35bf-4fec-981f-901931fa2f0c None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.336534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d2969-35bf-4fec-981f-901931fa2f0c None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.336819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d2969-35bf-4fec-981f-901931fa2f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.360601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-403d2969-35bf-4fec-981f-901931fa2f0c None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:12.362114 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-403d2969-35bf-4fec-981f-901931fa2f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.374847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1251/2541] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:12.387650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.389836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.411134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:12.472960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7a301-51f2-4543-8c8f-6e90ed1098be None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.480650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.481644 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.482116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.482588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.495748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.497536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b491e97-729e-43dd-bf10-855b4c595c04 service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.512305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1291/2542] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:12.599245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.599245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.599608 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.601165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1252/2543] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:51:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:51:12.606718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206d73-a84f-40c0-8802-a48f87755d3b None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.607327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206d73-a84f-40c0-8802-a48f87755d3b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.607327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206d73-a84f-40c0-8802-a48f87755d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.626587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-af206d73-a84f-40c0-8802-a48f87755d3b None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:12.629737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-af206d73-a84f-40c0-8802-a48f87755d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.691217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1292/2544] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:12.703963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.705979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.723664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddae38-6706-4dfa-8b6b-e445bd6b4fc6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.741126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.741891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.742169 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.742437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.750558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.752084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3328f-0011-480c-af92-97e29a94ac20 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.760279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1253/2545] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:12.851362 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db61e3-3821-4701-8fba-f957ea52c055 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.851714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db61e3-3821-4701-8fba-f957ea52c055 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.852047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db61e3-3821-4701-8fba-f957ea52c055 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.853390 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1293/2546] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:51:12.860254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beb260-b7fc-4d9f-b270-9d8287b1c37d None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.860457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beb260-b7fc-4d9f-b270-9d8287b1c37d None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.860698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beb260-b7fc-4d9f-b270-9d8287b1c37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.881957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-51beb260-b7fc-4d9f-b270-9d8287b1c37d None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:12.883201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-51beb260-b7fc-4d9f-b270-9d8287b1c37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:12.895203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1254/2547] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:51:12.907338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db61e3-3821-4701-8fba-f957ea52c055 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:12.932380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db61e3-3821-4701-8fba-f957ea52c055 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:12.936868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:12.937606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:12.938013 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:12.938291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:12.951310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:12.953119 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:12.971032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1294/2548] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:16.117222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848387a-0294-4a51-bb46-908308f4feaa None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:16.117714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848387a-0294-4a51-bb46-908308f4feaa None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:16.117820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848387a-0294-4a51-bb46-908308f4feaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:16.138251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0848387a-0294-4a51-bb46-908308f4feaa None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:16.139374 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0848387a-0294-4a51-bb46-908308f4feaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:16.196428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1255/2549] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:51:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:16.218732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:16.220180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:16.288592 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d693c7-22e2-4be8-8b72-aeb1347f84cf service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:16.294086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:16.294696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:16.294977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:16.295247 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:16.304868 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:16.306509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:16.315195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1295/2550] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:51:25.595303 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813bf9d-6c32-41ad-a72c-148c832bbb85 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:25.595913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813bf9d-6c32-41ad-a72c-148c832bbb85 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:25.595913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813bf9d-6c32-41ad-a72c-148c832bbb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:25.603903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:25.616786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b813bf9d-6c32-41ad-a72c-148c832bbb85 None None] MFA Rules not processed for user `4279761de1464be38860432a46a89074`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:25.618031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b813bf9d-6c32-41ad-a72c-148c832bbb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:25.624906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1256/2551] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:26.311861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:35.611900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:36.408253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:36.418040 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61615c-4fac-48bb-8aa1-2a5248e5d515 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:36.421612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:36.422195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:36.422477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:36.422744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:36.424468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:36.485856 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:36.487869 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c57863-c58d-4311-b49f-ff36fc4b78b1 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:36.498066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1296/2552] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:38.981210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:38.982122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:38.982735 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:39.018947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] MFA Rules not processed for user `21fdd2fd30544e4e9e0bf184034106fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:39.020493 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:39.098191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1257/2553] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:51:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:43.715392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:43.715939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:43.716198 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:43.736901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:51:43.738666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:51:43.746309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1297/2554] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:51:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:51:43.758949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:43.772413 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4011fba-f68d-440a-98b2-4d820b58d7c3 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:43.777702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:43.778327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:43.778739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:43.779017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:43.788840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:43.790575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bf53f-aa0a-457b-aa9b-0fa04c348354 service swift] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:43.800200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1258/2555] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:51:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:51:48.438940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:51:48.448893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1f7fa-7039-42bb-92f7-8ee0de1665fd None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:51:48.452572 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:51:48.453171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:51:48.453452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:51:48.453721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:51:48.463327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:51:48.464777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:51:48.474051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1298/2556] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:51:53.796357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=31 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:51:58.470017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=39 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:33.419372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:33.419955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:33.419955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:33.438940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] MFA Rules not processed for user `7d3139d9e58f4b56a370661e2a5d4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:33.440166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:33.494004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1259/2557] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:41.562928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:41.564489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:41.629494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe0f-25cd-4b86-9d8d-96365a2eb758 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:41.634055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:41.634687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:41.634966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:41.635239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] PATH_INFO: `/v3/users/21fdd2fd30544e4e9e0bf184034106fa` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:41.636451 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fdd2fd30544e4e9e0bf184034106fa)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:41.638372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:41.683433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fdd2fd30544e4e9e0bf184034106fa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:41.684128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fdd2fd30544e4e9e0bf184034106fa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:41.691692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fdd2fd30544e4e9e0bf184034106fa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:41.698685 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fdd2fd30544e4e9e0bf184034106fa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:41.711059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5b781f13-4c9a-411a-a02c-e218dd29da8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fdd2fd30544e4e9e0bf184034106fa'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:41.716846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1299/2558] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:52:41 2022] DELETE /identity/v3/users/21fdd2fd30544e4e9e0bf184034106fa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:41.889243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:41.900544 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dcebb-027b-4bd2-b875-7903b91064e3 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:41.962458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:41.963104 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:41.963381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:41.963686 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] PATH_INFO: `/v3/projects/b3b756a7bd444bf3aec562b2717f5623` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:41.964860 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3b756a7bd444bf3aec562b2717f5623)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:41.966756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.011865 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3b756a7bd444bf3aec562b2717f5623', 'request_id': 'req-18b2a605-cd96-4e02-83b1-f843213d979b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:42.017313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3b756a7bd444bf3aec562b2717f5623', 'request_id': 'req-18b2a605-cd96-4e02-83b1-f843213d979b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:42.026113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3b756a7bd444bf3aec562b2717f5623', 'request_id': 'req-18b2a605-cd96-4e02-83b1-f843213d979b'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:42.038418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] The token cache is being invalidate because project b3b756a7bd444bf3aec562b2717f5623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:52:42.038711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:42.039377 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-18b2a605-cd96-4e02-83b1-f843213d979b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:52:42.040964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1260/2559] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:52:41 2022] DELETE /identity/v3/projects/b3b756a7bd444bf3aec562b2717f5623 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:42.046722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.047009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.047285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.068219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:42.069411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:42.126593 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1300/2560] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:42.135124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.135409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.135712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.155750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:42.156897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:42.164756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1261/2561] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:42.168931 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.178715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd39219-d50c-45c2-a577-6a02aabe9cf2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.183328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.183957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.184234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.184507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.185149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:52:42.185455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.187067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.198761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1301/2562] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:52:42.202974 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.213056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf19b-6b0d-445b-937f-ea9d82c5b851 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.217723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.218311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.218588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.218856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.219496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.221402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.247942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1262/2563] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:42 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:52:42.252285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.262833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3073b7-ec6a-4381-9237-19dc8a6ef08c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.266313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:42.321411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.322012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.322291 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.322560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.323220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.325270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.346220 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1302/2564] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:42 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:42.350395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.360817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c03149-82c3-426c-be0e-6f536439ce72 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.365601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.366197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.366477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.366748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.367384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.369023 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.370275 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:42.381452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1263/2565] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:42.384926 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.395325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85737f-0d43-45de-b139-bd1fff682ffe admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.399966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.400560 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.400835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.401102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.401733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.403304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554111f9-6572-4dca-aee6-daa335876a5f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.415112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1303/2566] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:52:42.419051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.429750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1547d53-9d60-490c-b259-cecc1eaefdf7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.434392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.435041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.435322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.435633 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] PATH_INFO: `/v3/projects/e78c5560d7874c7d9ae11d1a078090d9/users/f07443b7cf95488a9ccb84895a63b92e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.450442 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78c5560d7874c7d9ae11d1a078090d9, user_id=f07443b7cf95488a9ccb84895a63b92e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.453225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.466088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1264/2567] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:42 2022] PUT /identity/v3/projects/e78c5560d7874c7d9ae11d1a078090d9/users/f07443b7cf95488a9ccb84895a63b92e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:52:42.471110 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7390d0-468b-41b4-b7d0-2828cd5c4136 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.471397 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7390d0-468b-41b4-b7d0-2828cd5c4136 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.471703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7390d0-468b-41b4-b7d0-2828cd5c4136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.490420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-9a7390d0-468b-41b4-b7d0-2828cd5c4136 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:42.491686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9a7390d0-468b-41b4-b7d0-2828cd5c4136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:42.549290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1304/2568] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:42.559295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:42.561035 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:42.617545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d1340-6a77-4d19-85f2-fb26a66a40bc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:42.621197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:42.621780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:42.622055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:42.622322 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:42.631638 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:42.633059 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:42.642213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1265/2569] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:47.006194 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.007162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.007702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.057475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] MFA Rules not processed for user `f07443b7cf95488a9ccb84895a63b92e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:47.059176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:47.142107 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1305/2570] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:47.146956 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.157025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bdba4a-ca8f-41a4-9a90-3c7c529a1bff service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.162277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.162891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.163182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.163441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.164093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.166350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.199930 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1266/2571] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:47.202676 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.216211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03824c9-1704-4a1f-a4bd-137cd50cb978 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.234965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:47.284941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.285465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.285739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.286006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.286642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.288605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.310182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1306/2572] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:52:47 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:52:47.314119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.324881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032d5-02f3-40cd-bddb-b05d00b1eef5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.329590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.330189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.330463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.331281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.332020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.333657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.344664 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1267/2573] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:52:47.348906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.361637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc58a15-16f3-4167-b84f-925ca3cf8511 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.366954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.367666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.367950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.368229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] PATH_INFO: `/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8/users/ccb6e8b1176d4dfbae25371cbc811345/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.386544 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b28aae9d084d5ab6c031f27f29a0f8, user_id=ccb6e8b1176d4dfbae25371cbc811345, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.390212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.407837 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1307/2574] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:47 2022] PUT /identity/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8/users/ccb6e8b1176d4dfbae25371cbc811345/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:52:47.411905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.425681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4269-3c11-423d-8115-546575e12d36 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.488829 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.489447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.489721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.489991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.490629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.492513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf1c13-c5b1-4dbf-861e-1e06b7e6053c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.502961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1268/2575] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:52:47.507065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:47.517522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d0872-19cc-4e18-88f4-8676b0bbcac9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:47.526356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:47.527284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:47.528843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:47.528843 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] PATH_INFO: `/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8/users/ccb6e8b1176d4dfbae25371cbc811345/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:47.530217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b28aae9d084d5ab6c031f27f29a0f8, user_id=ccb6e8b1176d4dfbae25371cbc811345, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:47.533003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:47.547601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1308/2576] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:52:47 2022] PUT /identity/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8/users/ccb6e8b1176d4dfbae25371cbc811345/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:52:52.680154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e4708-399e-4560-9ec7-c9478be27769 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:52.681091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e4708-399e-4560-9ec7-c9478be27769 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:52.681714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e4708-399e-4560-9ec7-c9478be27769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:52.705116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a09e4708-399e-4560-9ec7-c9478be27769 None None] MFA Rules not processed for user `7813dcd78a38483682842ac199e1cd18`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:52.706648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a09e4708-399e-4560-9ec7-c9478be27769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:52.752790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:52.780809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1269/2577] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:53.890395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:53.893582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:53.969278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f15d8-ba83-4302-b767-324f7712bf1b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:53.970389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f15d8-ba83-4302-b767-324f7712bf1b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:53.970847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f15d8-ba83-4302-b767-324f7712bf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:54.022589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76543c5c-10a2-4452-bcd9-59f06541bc94 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:54.026936 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:54.027529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:54.027834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:54.028048 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:54.029813 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:54.036449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-633f15d8-ba83-4302-b767-324f7712bf1b None None] MFA Rules not processed for user `ccb6e8b1176d4dfbae25371cbc811345`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:54.037685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-633f15d8-ba83-4302-b767-324f7712bf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:54.121624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:54.123092 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:54.144304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1309/2578] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:54.167323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1270/2579] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:52:54.190285 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:54.191937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:54.211392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:52:54.279454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050bd07-8783-420e-bcbe-447e8865abca service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:54.284475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:54.285117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:54.285444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:54.285664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:54.360059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:54.362122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:54.378597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1310/2580] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:52:54.535073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:54.535667 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:54.536028 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:54.555995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:52:54.557208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:54.628120 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1271/2581] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:52:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:52:54.685703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:54.687321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:54.768909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3efbb-1f60-4938-9fb1-0141c0dbe238 service glance] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:54.774499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:54.775231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:54.775599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:54.775807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:54.785954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:54.787718 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c3ad-7853-4ca6-be04-d2510d205ca3 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:54.797031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1311/2582] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:52:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:52:59.127788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:52:59.139277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d48a-6531-4ba3-9329-98d4e985161a None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:52:59.143720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:52:59.144458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:52:59.144838 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:52:59.145207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:52:59.147147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:52:59.162879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:52:59.164531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66932e2c-76fc-4c41-b05d-fdfc8e8f2a88 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:52:59.176847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1272/2583] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:52:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:04.041297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:04.041297 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:04.041761 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:04.061719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] MFA Rules not processed for user `f07443b7cf95488a9ccb84895a63b92e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:04.063185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:04.071345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1312/2584] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:05.952720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:05.953266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:05.955147 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:05.978350 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:05.979847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:05.988068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1273/2585] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:53:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:06.001508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:06.014650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e638-8abc-4b44-982d-1f3805896699 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:06.020606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:06.021231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:06.021509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:06.021782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:06.030729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:06.032474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:06.042579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1313/2586] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:53:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:53:10.604952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:10.606980 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:10.693432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f5a65-c2b7-4f09-a9d3-ee6eac17142f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:10.697288 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:10.697911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:10.698192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:10.698464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:10.700812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:10.708795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=100 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:10.788807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:10.790278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:10.801046 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1274/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:16.038950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=85 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:20.795819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=24 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:20.804722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:20.814378 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a3b5d-91b3-4a69-964c-d54e3eda7452 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:20.818204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:20.818814 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:20.819089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:20.819357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:20.821221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:20.831919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:20.833606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:20.841748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1314/2588] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:29.859012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:29.862928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:29.882663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7d3a8-219a-448f-b1c5-bf7b5ee2b0ff service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:29.889519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:29.890158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:29.890447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:29.891284 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:29.893399 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:29.909064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:29.910760 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:29.922647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1275/2589] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:30.838051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=6 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:35.233465 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:35.242091 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93f4dc-3819-44b4-a1d9-c602e631fda3 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:35.245733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:35.246326 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:35.246600 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:35.246865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:35.248616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:35.309727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:35.311162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:35.321204 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1315/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:37.499098 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:37.509384 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ab83-9308-423d-b287-34b2e72040d8 service nova] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:37.513533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:37.514216 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:37.514568 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:37.514912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:37.525520 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:37.527176 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:37.547540 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1276/2591] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:38.619391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:38.621028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:38.638433 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02e616-ca14-4488-8125-426e9ccaad5d service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:38.644029 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:38.644669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:38.644950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:38.645228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] PATH_INFO: `/v3/users/7d3139d9e58f4b56a370661e2a5d4e5e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:38.646388 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3139d9e58f4b56a370661e2a5d4e5e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:38.648327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:38.700489 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3139d9e58f4b56a370661e2a5d4e5e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:38.701153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3139d9e58f4b56a370661e2a5d4e5e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:38.708950 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3139d9e58f4b56a370661e2a5d4e5e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:38.716413 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3139d9e58f4b56a370661e2a5d4e5e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:38.729303 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-585f3364-c50d-4cad-b95c-6edddf26f40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3139d9e58f4b56a370661e2a5d4e5e'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:38.735577 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1316/2592] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:53:38 2022] DELETE /identity/v3/users/7d3139d9e58f4b56a370661e2a5d4e5e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:53:38.986473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:38.998741 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff09b-d9a9-44bc-ba6e-a71709280e78 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.079557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.080623 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.080906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.081179 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] PATH_INFO: `/v3/projects/0fb78a439e20439f8d93a4f7df3f28f8` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.082625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb78a439e20439f8d93a4f7df3f28f8)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.084640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.139298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb78a439e20439f8d93a4f7df3f28f8', 'request_id': 'req-d18923c4-be27-4e2c-9176-24f855b55c32'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:39.145814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb78a439e20439f8d93a4f7df3f28f8', 'request_id': 'req-d18923c4-be27-4e2c-9176-24f855b55c32'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:39.153999 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb78a439e20439f8d93a4f7df3f28f8', 'request_id': 'req-d18923c4-be27-4e2c-9176-24f855b55c32'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:39.165795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] The token cache is being invalidate because project 0fb78a439e20439f8d93a4f7df3f28f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:53:39.166092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:39.166748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-d18923c4-be27-4e2c-9176-24f855b55c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:53:39.168491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1277/2593] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:53:38 2022] DELETE /identity/v3/projects/0fb78a439e20439f8d93a4f7df3f28f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:53:39.173382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.173671 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.173964 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.194925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:39.196225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:39.258513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1317/2594] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:39.267257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.267593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.267976 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.289171 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:39.290395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:39.299062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1278/2595] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:39.303117 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.313832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2666e3c-b7ca-4090-bd27-61b30ad489f0 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.318521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.319120 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.319393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.319715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.320350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:53:39.320646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.322263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.323873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:39.340278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1318/2596] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:53:39.341570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.351064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ad3d4-e891-4b45-869a-ba94b941f6b8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.355935 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.356914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.356914 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.357130 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.357661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.359582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.388816 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1279/2597] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:53:39 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:53:39.393764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.403657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3ffbf-5db3-4101-a35e-e350978d293f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.469838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.470537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.470816 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.471083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.471744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.473641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.495863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1319/2598] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:53:39 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:53:39.499798 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.511139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac9bb2-9b3e-458b-927a-90cb8ddd672b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.524482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.525166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.526774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.531607 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.532327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.533951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.574942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.576878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:39.690300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1280/2599] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:53:39.694026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.703963 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185e09c-ecbb-4730-a639-a046ae1daf73 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.706406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440828f1-65aa-41c8-a2db-1daae6b2e5d2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.710848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.711561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.711848 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.711998 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.712162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.712720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.713163 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.714271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:39.715364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.716160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.717990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.731684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1281/2600] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:53:39.734080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.736058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.738900 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15716e2b-269e-4298-9db7-3ffae46f33e6 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.750949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c947270-7c3c-432d-a567-b75d797c6c9f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:39.754628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1320/2601] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:53:39.757191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:39.757920 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.758204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.758476 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] PATH_INFO: `/v3/projects/625652cfa47248faac8c1f7759c56a14/users/d8b9d2b042c04927b9115085fcb4ba40/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.773034 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:39.776912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625652cfa47248faac8c1f7759c56a14, user_id=d8b9d2b042c04927b9115085fcb4ba40, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:39.780012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:39.802744 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1282/2602] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:53:39 2022] PUT /identity/v3/projects/625652cfa47248faac8c1f7759c56a14/users/d8b9d2b042c04927b9115085fcb4ba40/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:53:39.808146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:39.808450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:39.808779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:39.830610 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:39.832886 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:39.977098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1321/2603] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:53:39.983330 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:39.987192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:40.071835 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb87ef1-0d09-4963-bded-4587f3cd7fa7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:40.076873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:40.077733 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:40.078390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:40.078734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:40.090696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:40.092606 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cbbfd-c351-430e-83f6-377dad84b084 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:40.106996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1283/2604] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:53:43.387490 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:43.389141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:43.406573 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:53:43.502218 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b29bb7-ecef-4001-98c6-c2d90ffbb45f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:43.511109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:43.511885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:43.512369 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:43.512626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:43.515171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:43.591327 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:43.592994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:43.603059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1322/2605] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:44.647258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42ce7-0bb8-420d-8154-111f4918eceb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:44.647817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42ce7-0bb8-420d-8154-111f4918eceb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:44.647817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b42ce7-0bb8-420d-8154-111f4918eceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:44.680704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c0b42ce7-0bb8-420d-8154-111f4918eceb None None] MFA Rules not processed for user `d8b9d2b042c04927b9115085fcb4ba40`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:44.681600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c0b42ce7-0bb8-420d-8154-111f4918eceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:44.774349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1284/2606] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:44.790824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:44.805011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a309ce-dac9-4509-8571-e6a88d2bd7e2 service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:44.811070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:44.811854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:44.812161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:44.812427 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:44.821228 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:44.822857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:44.833103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1323/2607] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:46.184409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:46.184953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:46.184953 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:46.204757 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] MFA Rules not processed for user `d8b9d2b042c04927b9115085fcb4ba40`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:53:46.206118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:46.213899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1285/2608] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:53:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:53:46.542334 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:46.543809 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:46.556289 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb36de6-c0b8-4f5c-9f70-997b8d1af0c4 service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:46.561879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:46.563224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:46.563702 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:46.563980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:46.569300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:46.582748 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:46.585070 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:46.595122 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1324/2609] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:53:46.956105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:46.958400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:47.035814 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510c8e5-d814-445b-98aa-a81635ae7430 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:47.040832 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:47.041440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:47.041717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:47.041986 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:47.043734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:47.056022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:47.057572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2188b-1906-4883-9dbc-2ba31e0f3773 service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:47.070138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1286/2610] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:55.552608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:53:55.566710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b02eb-69dc-4f58-864d-51ef263643e7 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:53:55.572688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:53:55.573237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:53:55.573511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:53:55.573807 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:53:55.575732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:53:55.588517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:53:55.590100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4724a5fa-a888-4dad-9889-1a7c1d3ffbd5 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:53:55.601097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1325/2611] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:53:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:53:57.064301 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:05.596611 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=70 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:10.241831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a275e-c14b-4849-837b-77118f71d832 None None] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.242456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a275e-c14b-4849-837b-77118f71d832 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.242588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a275e-c14b-4849-837b-77118f71d832 None None] PATH_INFO: `/` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.244084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1287/2612] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:54:10.250493 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4e7c-09a1-49e4-9e87-70357e86785a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.250968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4e7c-09a1-49e4-9e87-70357e86785a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.251093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4e7c-09a1-49e4-9e87-70357e86785a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.271471 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a55e4e7c-09a1-49e4-9e87-70357e86785a None None] MFA Rules not processed for user `c18cb9acd43548ed9c8efe69f77b8df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:10.272728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a55e4e7c-09a1-49e4-9e87-70357e86785a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:10.279268 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1326/2613] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:10.294316 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a275e-c14b-4849-837b-77118f71d832 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:10.305463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a275e-c14b-4849-837b-77118f71d832 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:10.309348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:10.310188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.310531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.310913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.320780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:10.322213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7842f0-8210-4e45-bcc9-4a8971dfef19 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:10.331346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1288/2614] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:54:10.433759 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0b4d-a4ba-4550-976c-b58cec54129b None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.434043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0b4d-a4ba-4550-976c-b58cec54129b None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.434338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b0b4d-a4ba-4550-976c-b58cec54129b None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.435763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1327/2615] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:54:10.440513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219442-fdaf-49f6-a9e8-429c53f3929f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.440809 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219442-fdaf-49f6-a9e8-429c53f3929f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.441102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219442-fdaf-49f6-a9e8-429c53f3929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.461003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-81219442-fdaf-49f6-a9e8-429c53f3929f None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:10.462151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-81219442-fdaf-49f6-a9e8-429c53f3929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:10.472056 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1289/2616] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:10.482222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0b4d-a4ba-4550-976c-b58cec54129b None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:10.493178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b0b4d-a4ba-4550-976c-b58cec54129b None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:10.496828 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:10.497420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:10.497698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:10.497996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:10.508779 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:10.510210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:10.521841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1328/2617] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:12.754949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8d9d-1285-4d04-b089-c80f40919fc4 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:12.756148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8d9d-1285-4d04-b089-c80f40919fc4 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:12.756604 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8d9d-1285-4d04-b089-c80f40919fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:12.777072 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4e1c8d9d-1285-4d04-b089-c80f40919fc4 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:12.778586 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4e1c8d9d-1285-4d04-b089-c80f40919fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:12.848632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1290/2618] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:12.891750 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:12.893275 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:12.982875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac4c3a-4fd3-4a9d-8b52-e7375b84c329 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:12.989712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:12.990330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:12.990889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:12.991307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:13.000296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:13.002159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2fbe38-fc21-4420-bc02-f2ac954512bd service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:13.019065 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1329/2619] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:54:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:54:14.748166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214313-56e9-403a-9773-92a4c64c85fc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:14.748791 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214313-56e9-403a-9773-92a4c64c85fc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:14.748912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3214313-56e9-403a-9773-92a4c64c85fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:14.770788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e3214313-56e9-403a-9773-92a4c64c85fc None None] MFA Rules not processed for user `4279761de1464be38860432a46a89074`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:14.772557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e3214313-56e9-403a-9773-92a4c64c85fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:14.842062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1291/2620] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:54:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:54:23.013350 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=41 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:24.834364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=59 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:29.145776 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:29.146321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:29.146436 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] PATH_INFO: `/` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:29.148082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1330/2621] 10.222.0.27 () {54 vars in 921 bytes} [Wed Aug 24 17:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:54:29.153010 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ae31-45f2-4e67-89e9-ae4e12f4879f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:29.153448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ae31-45f2-4e67-89e9-ae4e12f4879f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:29.153557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ae31-45f2-4e67-89e9-ae4e12f4879f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:29.174405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-fdd1ae31-45f2-4e67-89e9-ae4e12f4879f None None] MFA Rules not processed for user `1a6e9e58daeb47abb6277e7c2bf6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:29.175647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fdd1ae31-45f2-4e67-89e9-ae4e12f4879f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:29.185299 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1292/2622] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Aug 24 17:54:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:29.196579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:29.198276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:29.268961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af66b4-59ed-4670-85af-ab6f62b3ea54 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:29.272911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:29.273518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:29.273797 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:29.274069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:29.285968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:29.287416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:29.299584 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1331/2623] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:54:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:54:31.072915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a15-9509-4dd8-a63b-6cc4cdbdbce3 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:31.073416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a15-9509-4dd8-a63b-6cc4cdbdbce3 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:31.073532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a15-9509-4dd8-a63b-6cc4cdbdbce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:31.093536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-79fa5a15-9509-4dd8-a63b-6cc4cdbdbce3 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:31.094737 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-79fa5a15-9509-4dd8-a63b-6cc4cdbdbce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:31.101847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1293/2624] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:54:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:31.114140 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:31.125332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647113-d0ee-43c4-8c97-73a7ed193095 service glance] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:31.130470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:31.131051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:31.131333 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:31.131645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:31.140684 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:31.142346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:31.150424 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1332/2625] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:54:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:54:41.087831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=6 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:41.147142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=32 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:54:47.355922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c469-04f6-4fc8-8b49-82344671ac20 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:47.356886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c469-04f6-4fc8-8b49-82344671ac20 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:47.357394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c469-04f6-4fc8-8b49-82344671ac20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:47.378074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8a12c469-04f6-4fc8-8b49-82344671ac20 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:47.379703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8a12c469-04f6-4fc8-8b49-82344671ac20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:47.387461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1294/2626] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:54:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:47.413893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:54:47.425605 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b160b-5ae3-4e75-a4ad-ca30245a2b3a service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:54:47.431447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:54:47.432454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:47.433024 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:47.433887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:47.445615 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:54:47.448172 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4431dd-a02b-4120-855a-1e7a6cd8d34a service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:54:47.459603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1333/2627] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:54:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:54:53.610480 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a86b8-3912-4586-af18-486391306163 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:54:53.611061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a86b8-3912-4586-af18-486391306163 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:54:53.611061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a86b8-3912-4586-af18-486391306163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:54:53.631641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b22a86b8-3912-4586-af18-486391306163 None None] MFA Rules not processed for user `d8b9d2b042c04927b9115085fcb4ba40`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:54:53.632893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b22a86b8-3912-4586-af18-486391306163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:54:53.640573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1295/2628] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:54:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:54:57.454727 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=6 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:03.625669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=6 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:29.791461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:29.791965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:29.792064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:29.811589 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:29.812794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:29.819973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1334/2629] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:55:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:29.841657 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a86b8-3912-4586-af18-486391306163 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:29.853831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a86b8-3912-4586-af18-486391306163 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:29.858711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e55898-4469-4390-b802-c44a36732807 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:29.859304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55898-4469-4390-b802-c44a36732807 service swift] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:29.860243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55898-4469-4390-b802-c44a36732807 service swift] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:29.860518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e55898-4469-4390-b802-c44a36732807 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:29.868462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e55898-4469-4390-b802-c44a36732807 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:29.870029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e55898-4469-4390-b802-c44a36732807 service swift] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:29.878157 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1296/2630] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:55:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:55:33.943007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:33.944561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:33.961153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04929d6b-7e3b-43bc-aaf0-3b0d99b5ef02 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:33.966738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:33.967534 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:33.967922 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:33.968331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:33.970256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:33.993429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:33.995877 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:34.021540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1335/2631] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:36.200727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e55898-4469-4390-b802-c44a36732807 service swift] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:36.211867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e55898-4469-4390-b802-c44a36732807 service swift] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:36.216149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:36.216756 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:36.217031 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:36.217297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:36.219740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:36.235600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:36.237574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:36.248012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1297/2632] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:37.558905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:37.570162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d67f2-0206-45fd-a8eb-7b081389f48f service placement] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:37.575863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:37.576479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:37.578533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:37.578825 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:37.580927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:37.595127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:37.597080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:37.619067 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1336/2633] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:37.648705 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:37.651033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:37.668884 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5a87f-3646-4411-8bff-8013b46b551a service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:37.675156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:37.676055 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:37.676537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:37.677093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] PATH_INFO: `/v3/users/f07443b7cf95488a9ccb84895a63b92e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:37.678392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07443b7cf95488a9ccb84895a63b92e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:37.680488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:37.737396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07443b7cf95488a9ccb84895a63b92e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:37.738122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07443b7cf95488a9ccb84895a63b92e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:37.745856 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07443b7cf95488a9ccb84895a63b92e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:37.760011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07443b7cf95488a9ccb84895a63b92e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:37.767588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07443b7cf95488a9ccb84895a63b92e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:37.773928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1298/2634] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:37 2022] DELETE /identity/v3/users/f07443b7cf95488a9ccb84895a63b92e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:37.945842 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:37.957561 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8d47e-7411-4c40-8e42-a8dac2818a18 service placement] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:38.021017 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:38.021663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:38.021945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:38.022221 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] PATH_INFO: `/v3/projects/e78c5560d7874c7d9ae11d1a078090d9` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:38.024232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78c5560d7874c7d9ae11d1a078090d9)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:38.026428 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:38.080647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78c5560d7874c7d9ae11d1a078090d9', 'request_id': 'req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.087703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78c5560d7874c7d9ae11d1a078090d9', 'request_id': 'req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.099072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78c5560d7874c7d9ae11d1a078090d9', 'request_id': 'req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.118351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] The token cache is being invalidate because project e78c5560d7874c7d9ae11d1a078090d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:38.118651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.119340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.124578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1337/2635] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:37 2022] DELETE /identity/v3/projects/e78c5560d7874c7d9ae11d1a078090d9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:38.128388 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:38.130266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:38.147000 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083d416-d74f-4f55-9b10-105ec560f226 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:38.216895 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:38.217058 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:38.217358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:38.217697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] PATH_INFO: `/v3/users/ccb6e8b1176d4dfbae25371cbc811345` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:38.219864 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb6e8b1176d4dfbae25371cbc811345)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:38.221874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:38.284324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb6e8b1176d4dfbae25371cbc811345'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.285943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb6e8b1176d4dfbae25371cbc811345'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.292595 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:38.298173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb6e8b1176d4dfbae25371cbc811345'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.317062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb6e8b1176d4dfbae25371cbc811345'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.325121 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb6e8b1176d4dfbae25371cbc811345'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.333897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1299/2636] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:38 2022] DELETE /identity/v3/users/ccb6e8b1176d4dfbae25371cbc811345 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:55:38.562881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:38.564990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:38.581455 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0394e-9e80-4f3b-9d8c-60b5fe46fbff admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:38.643128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:38.650778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:38.651076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:38.651393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:38.652191 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] PATH_INFO: `/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:38.654899 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b28aae9d084d5ab6c031f27f29a0f8)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:38.657998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:38.722098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b28aae9d084d5ab6c031f27f29a0f8', 'request_id': 'req-11271c5a-2799-4dc8-b120-13c457900e58'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.730185 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b28aae9d084d5ab6c031f27f29a0f8', 'request_id': 'req-11271c5a-2799-4dc8-b120-13c457900e58'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.742371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b28aae9d084d5ab6c031f27f29a0f8', 'request_id': 'req-11271c5a-2799-4dc8-b120-13c457900e58'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.755368 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] The token cache is being invalidate because project 27b28aae9d084d5ab6c031f27f29a0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:38.755763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.756508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:38.759638 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1338/2637] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:38 2022] DELETE /identity/v3/projects/27b28aae9d084d5ab6c031f27f29a0f8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:40.269123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:40.270626 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:40.284068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8cd58-0286-4237-98d4-7c4015043188 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:40.340578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:40.341214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.341496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.341833 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] PATH_INFO: `/v3/users/7813dcd78a38483682842ac199e1cd18` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.342967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7813dcd78a38483682842ac199e1cd18)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:40.345119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:40.391391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7813dcd78a38483682842ac199e1cd18'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.392097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7813dcd78a38483682842ac199e1cd18'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.399084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7813dcd78a38483682842ac199e1cd18'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.410893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7813dcd78a38483682842ac199e1cd18'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.419074 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-8d221d98-8e39-4b46-8bba-60cf397ab136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7813dcd78a38483682842ac199e1cd18'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.426151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1300/2638] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:55:40 2022] DELETE /identity/v3/users/7813dcd78a38483682842ac199e1cd18 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:40.599071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:40.600875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:40.617233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11271c5a-2799-4dc8-b120-13c457900e58 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:40.675782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:40.676419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.676701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.676977 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] PATH_INFO: `/v3/projects/2fe3fa5e280d4fbd8a26cac05fe8aaa9` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.678115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe3fa5e280d4fbd8a26cac05fe8aaa9)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:40.680072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:40.726213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe3fa5e280d4fbd8a26cac05fe8aaa9', 'request_id': 'req-08ea51e6-a042-4524-9a8f-b42999395045'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.731624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe3fa5e280d4fbd8a26cac05fe8aaa9', 'request_id': 'req-08ea51e6-a042-4524-9a8f-b42999395045'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.740118 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe3fa5e280d4fbd8a26cac05fe8aaa9', 'request_id': 'req-08ea51e6-a042-4524-9a8f-b42999395045'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.751745 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] The token cache is being invalidate because project 2fe3fa5e280d4fbd8a26cac05fe8aaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:55:40.752028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.752648 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-08ea51e6-a042-4524-9a8f-b42999395045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:55:40.754139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1339/2639] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:55:40 2022] DELETE /identity/v3/projects/2fe3fa5e280d4fbd8a26cac05fe8aaa9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:40.759039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.759462 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.759701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.778991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:40.780155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:40.819557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:40.839250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1301/2640] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:40.848491 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.848780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.849058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.869116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:40.870336 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:40.878375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1340/2641] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:40.882738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:40.892461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589d085-ffa3-4ffb-8b9e-537541b32815 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:40.897002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:40.897593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.897868 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.898138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.898767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:55:40.899088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:40.900701 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:40.912646 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1302/2642] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:55:40.916939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:40.926884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5ed2e-f79c-4498-97fa-3c9b8d5d3293 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:40.931743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:40.932348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:40.932626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:40.932895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:40.933515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:40.935411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:40.944909 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:40.964636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1341/2643] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:55:40 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:40.969411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:40.979373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73dcbb-4b7c-48d3-86cc-68411a16c47f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:41.035326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:41.035877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.036154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.036423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.037045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:41.038958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:41.061309 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1303/2644] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:55:40 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:55:41.065520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:41.075840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11777cf-a475-4213-8acd-12891ed1fef1 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:41.081508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:41.082105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.082387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.082662 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.083299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:41.084939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:41.095631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1342/2645] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:41.099610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:41.110298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0276221-3d4c-4499-987e-b1d32eb0ee89 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:41.114940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:41.115592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.115877 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.116150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.116781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:41.118369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1ae2e-83ff-4bdc-8164-6f34c07f7327 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:41.133440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1304/2646] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:55:41.137914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:41.151199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319b6a0-2209-4dd9-b830-8ce6f6cae57b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:41.156677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:41.157335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.157617 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.157893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] PATH_INFO: `/v3/projects/1422bbfed9414b8cada1656aed195a31/users/1daca2cfac58487189f1fe8bf4bcb130/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.172284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1422bbfed9414b8cada1656aed195a31, user_id=1daca2cfac58487189f1fe8bf4bcb130, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:41.174929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:41.188994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1343/2647] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:55:41 2022] PUT /identity/v3/projects/1422bbfed9414b8cada1656aed195a31/users/1daca2cfac58487189f1fe8bf4bcb130/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:55:41.194291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8bb5-1d98-43d0-a1e4-eed799def849 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.194580 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8bb5-1d98-43d0-a1e4-eed799def849 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.194857 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f8bb5-1d98-43d0-a1e4-eed799def849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.214311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4d1f8bb5-1d98-43d0-a1e4-eed799def849 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:41.215564 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4d1f8bb5-1d98-43d0-a1e4-eed799def849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:41.275334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1305/2648] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:41.285714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:41.287402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:41.344233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f183a0b-8b5c-4508-80f6-e2fd6e723bc4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:41.347979 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:41.348588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:41.348865 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:41.349137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:41.358108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:41.359581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:41.369105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1344/2649] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:45.652704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca43-f5c2-45f3-aef1-74559a1064db None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:45.653245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca43-f5c2-45f3-aef1-74559a1064db None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:45.653245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8ca43-f5c2-45f3-aef1-74559a1064db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:45.690562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a3b8ca43-f5c2-45f3-aef1-74559a1064db None None] MFA Rules not processed for user `1daca2cfac58487189f1fe8bf4bcb130`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:45.692091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a3b8ca43-f5c2-45f3-aef1-74559a1064db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:45.742528 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:45.771090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1306/2650] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:45.785687 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:45.787904 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:45.870836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cbf77-3424-4f95-96b4-96db20b4c51c service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:45.877811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:45.878449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:45.878731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:45.879006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:45.890058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:45.891826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:45.901764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1345/2651] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:47.702281 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0f8f-ca3c-4102-8b3a-3ad64a7c8f59 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:47.704707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0f8f-ca3c-4102-8b3a-3ad64a7c8f59 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:47.705247 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0f8f-ca3c-4102-8b3a-3ad64a7c8f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:47.729797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e80f0f8f-ca3c-4102-8b3a-3ad64a7c8f59 None None] MFA Rules not processed for user `1daca2cfac58487189f1fe8bf4bcb130`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:47.731036 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e80f0f8f-ca3c-4102-8b3a-3ad64a7c8f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:47.739045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1307/2652] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:51.526016 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:51.527558 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:51.540559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833c6d-f9b8-4333-b969-6707ec5741dd service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:51.544222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:51.544836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:51.545114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:51.545386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:51.548081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:51.596519 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:55:51.616154 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:51.617889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:51.628552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1346/2653] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:56.278241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:56.278772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:56.278772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:56.297680 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] MFA Rules not processed for user `d8b9d2b042c04927b9115085fcb4ba40`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:55:56.298876 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:56.360624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1308/2654] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:55:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:55:56.678994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:56.687693 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344ab1-9e62-40ca-890b-db7d6143fe46 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:56.691373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:56.692004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:56.692283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:56.692554 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:56.694282 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:56.703966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:56.705389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:56.713363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1347/2655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:55:57.125625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:55:57.136457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9590d-6e63-48bf-b40d-1bdb302c2837 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:55:57.140105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:55:57.140736 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:55:57.141029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:55:57.141286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:55:57.143027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:55:57.155745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:55:57.157200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:55:57.167468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1309/2656] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:56:04.415581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:04.417689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:04.516696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ede03b-606f-479f-88fc-7d89e803ba97 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:04.523827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:04.524477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:04.524815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:04.525095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] PATH_INFO: `/v3/users/c75bb50ad7704e66983959d338b48e27` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:04.526244 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75bb50ad7704e66983959d338b48e27)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:04.528290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:04.584456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75bb50ad7704e66983959d338b48e27'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.585380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75bb50ad7704e66983959d338b48e27'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.597084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75bb50ad7704e66983959d338b48e27'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.606145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75bb50ad7704e66983959d338b48e27'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.622696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75bb50ad7704e66983959d338b48e27'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:04.640667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1348/2657] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:04 2022] DELETE /identity/v3/users/c75bb50ad7704e66983959d338b48e27 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:05.176428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.247310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32861f63-c2ae-4f00-af40-8fe4a583a023 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.316743 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.317391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.317673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.317946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] PATH_INFO: `/v3/projects/79ef0ce1ee884e9d80d4200d40b7b6c2` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.319125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ef0ce1ee884e9d80d4200d40b7b6c2)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.321196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.360126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:05.379081 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ef0ce1ee884e9d80d4200d40b7b6c2', 'request_id': 'req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.385315 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ef0ce1ee884e9d80d4200d40b7b6c2', 'request_id': 'req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.395001 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ef0ce1ee884e9d80d4200d40b7b6c2', 'request_id': 'req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.409407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] The token cache is being invalidate because project 79ef0ce1ee884e9d80d4200d40b7b6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:05.409925 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.410719 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.413014 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1310/2658] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:05 2022] DELETE /identity/v3/projects/79ef0ce1ee884e9d80d4200d40b7b6c2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:05.416732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.418325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.434399 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a1779-36d0-4a4f-ae4f-ced9ae1ff4e0 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.471901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:05.507102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.507836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.508834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.508834 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] PATH_INFO: `/v3/users/3133926f799346b48699867982fb9a6b` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.509831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3133926f799346b48699867982fb9a6b)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.511841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.579499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3133926f799346b48699867982fb9a6b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.581097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3133926f799346b48699867982fb9a6b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.590794 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3133926f799346b48699867982fb9a6b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.599808 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3133926f799346b48699867982fb9a6b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.613015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3133926f799346b48699867982fb9a6b'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:05.619043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1349/2659] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:56:05 2022] DELETE /identity/v3/users/3133926f799346b48699867982fb9a6b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:05.818040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:05.820307 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:05.837729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522fcdf-9f1b-41a3-b6d7-195ecdf5a7d5 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:05.920396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:05.922061 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:05.922433 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:05.922939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] PATH_INFO: `/v3/projects/3aabd6541e7e4d429574cedba7e6e607` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:05.924913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aabd6541e7e4d429574cedba7e6e607)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:05.931977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:05.995745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aabd6541e7e4d429574cedba7e6e607', 'request_id': 'req-282f480c-b3cf-46d6-8e83-b6c7a14e8906'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.002346 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aabd6541e7e4d429574cedba7e6e607', 'request_id': 'req-282f480c-b3cf-46d6-8e83-b6c7a14e8906'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.014533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aabd6541e7e4d429574cedba7e6e607', 'request_id': 'req-282f480c-b3cf-46d6-8e83-b6c7a14e8906'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.038005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] The token cache is being invalidate because project 3aabd6541e7e4d429574cedba7e6e607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:56:06.038310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.039052 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:56:06.056296 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1311/2660] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:56:05 2022] DELETE /identity/v3/projects/3aabd6541e7e4d429574cedba7e6e607 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:56:11.319043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:11.320513 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:11.428153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9a62-b711-4b05-b860-07badb944e50 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:11.447546 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:11.494815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:11.495345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:11.495667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:11.495942 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:11.498211 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:11.509682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:11.511255 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efde28b-2069-4855-a332-ba20c4c57f1d service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:11.522051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1350/2661] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:16.032611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:21.517633 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=21 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:26.163178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:26.164596 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:26.240435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f480c-b3cf-46d6-8e83-b6c7a14e8906 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:26.243987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:26.244581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:26.244855 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:26.245116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:26.246836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:26.304095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:26.305458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:26.314327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1312/2662] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:56:34.172206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087e57-c3a2-43ae-9fa8-2f743e18a0f2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:34.172714 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087e57-c3a2-43ae-9fa8-2f743e18a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:34.172819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087e57-c3a2-43ae-9fa8-2f743e18a0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:34.192072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-bc087e57-c3a2-43ae-9fa8-2f743e18a0f2 None None] MFA Rules not processed for user `4279761de1464be38860432a46a89074`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:56:34.193271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-bc087e57-c3a2-43ae-9fa8-2f743e18a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:34.200925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1351/2663] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:56:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:56:36.310454 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=57 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:44.186608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:56:50.476131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:56:50.477575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:50.557126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33676926-8c0b-4343-a8a8-57758bd0c8ad service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:56:50.561146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:56:50.561746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:56:50.562029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:56:50.562298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:56:50.564090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:56:50.627457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:56:50.628984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:56:50.639123 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1313/2664] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:57:00.634152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=59 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:06.266039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:06.266545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:06.266655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:06.285557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] MFA Rules not processed for user `d8b9d2b042c04927b9115085fcb4ba40`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:57:06.286770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:06.342239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1352/2665] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:57:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:57:16.335059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=25 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:57:54.722703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:57:54.724167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:54.736013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e832fa-009a-4f31-943f-55ef8264b95a service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:57:54.739573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:57:54.740165 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:57:54.740439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:57:54.740707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:57:54.742398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:57:54.831334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:57:54.832732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:57:54.853274 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1314/2666] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:57:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:00.930392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:00.932472 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:00.945018 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e80ab5-fd7c-4b38-953e-2ec6bfb9608f None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:00.949217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:00.949932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:00.950212 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:00.950478 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:00.952208 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:00.965482 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:00.966903 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:00.977517 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1353/2667] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:04.850581 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=46 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:10.972840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=6 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:15.576488 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:15.578026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:15.646902 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be93361-5ed1-4a74-9776-1cbb29f6b4af service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:15.651358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:15.652045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:15.652463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:15.652808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] PATH_INFO: `/v3/users/d8b9d2b042c04927b9115085fcb4ba40` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:15.654003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b9d2b042c04927b9115085fcb4ba40)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:15.655943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:15.701222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b9d2b042c04927b9115085fcb4ba40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:15.701882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b9d2b042c04927b9115085fcb4ba40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:15.709136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b9d2b042c04927b9115085fcb4ba40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:15.720661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b9d2b042c04927b9115085fcb4ba40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:15.727913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-46f2c0eb-10ba-4d4a-baed-9cd871d08507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b9d2b042c04927b9115085fcb4ba40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:15.734154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1315/2668] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:15 2022] DELETE /identity/v3/users/d8b9d2b042c04927b9115085fcb4ba40 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:15.908150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:15.919719 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020744fe-d83c-46fb-97c1-d3a7849a3e1c service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:15.977743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:15.978376 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:15.978657 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:15.978929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] PATH_INFO: `/v3/projects/625652cfa47248faac8c1f7759c56a14` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:15.980068 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625652cfa47248faac8c1f7759c56a14)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:15.981968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.027315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625652cfa47248faac8c1f7759c56a14', 'request_id': 'req-178f1b4b-7cce-4b2d-8553-c86505e11037'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.032872 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625652cfa47248faac8c1f7759c56a14', 'request_id': 'req-178f1b4b-7cce-4b2d-8553-c86505e11037'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.040925 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625652cfa47248faac8c1f7759c56a14', 'request_id': 'req-178f1b4b-7cce-4b2d-8553-c86505e11037'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.051733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] The token cache is being invalidate because project 625652cfa47248faac8c1f7759c56a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:16.052240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.052873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-178f1b4b-7cce-4b2d-8553-c86505e11037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:16.054418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1354/2669] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:15 2022] DELETE /identity/v3/projects/625652cfa47248faac8c1f7759c56a14 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:16.060124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.060411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.060690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.120458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:16.121697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.167069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:16.185674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1316/2670] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:16.193370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.193670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.194295 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.214500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:16.215770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.223953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1355/2671] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:16.228101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.237787 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b5a8-8e0f-4181-88b6-b796d9f16cd9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.242439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.243589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.243867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.244134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.244790 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:16.245088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.246702 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.258032 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1317/2672] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:16.262724 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.272036 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345517ec-328a-40fe-bfa9-acac6b0c5979 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.276668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.277258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.277595 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.277810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.278441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.280379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.316983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1356/2673] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:16.321448 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.331241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678404-6e5e-41cc-b8f1-826e57e395d4 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.386449 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.387053 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.387334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.387670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.388300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.390199 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.417508 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1318/2674] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:16.422216 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.433071 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27227f-0847-4020-bdba-2c2fb58af573 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.438339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.438933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.439226 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.439499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.440141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.441729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.452222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1357/2675] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:16.456292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.466609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115327a6-c762-4ea1-ac53-0f6de70673a7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.471499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.472108 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.472385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.472656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.473292 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.474906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb7ab3-028d-46ab-bd2b-c88557559a04 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.485990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1319/2676] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:16.490094 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.500704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f859-c0d0-48c0-b621-f24e39496f2a admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.505298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.505938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.506224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.506497 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] PATH_INFO: `/v3/projects/c3f39580ab584e8ab9aa3a4d9e86d5d9/users/1183cb7ade7b4758b9e6a106b7980355/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.520930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f39580ab584e8ab9aa3a4d9e86d5d9, user_id=1183cb7ade7b4758b9e6a106b7980355, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.523738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.536960 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1358/2677] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:16 2022] PUT /identity/v3/projects/c3f39580ab584e8ab9aa3a4d9e86d5d9/users/1183cb7ade7b4758b9e6a106b7980355/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:16.548670 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.548967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.549264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.576848 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] MFA Rules not processed for user `1183cb7ade7b4758b9e6a106b7980355`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:16.578060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.641181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1320/2678] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:16.652945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:16.654647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:16.675257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:16.730810 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754b489-4786-4674-9a6b-4f2f1f0e0096 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:16.736457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:16.737062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:16.737343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:16.737618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:16.745980 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:16.747582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:16.756293 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1359/2679] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:22.073472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:22.075153 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:22.152082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:22.166782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db0330-648e-4c20-a89a-194deed9f515 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:22.172593 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:22.173828 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:22.174339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:22.174619 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:22.176745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:22.187991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:22.189497 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341babc-de4c-413b-9e95-212fa1ed4a74 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:22.198912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1321/2680] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:58:25.093893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.153371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122ca0b-cd41-4c37-847e-3d1f300ee44b service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.157845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.158464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.158742 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.159012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] PATH_INFO: `/v3/users/1183cb7ade7b4758b9e6a106b7980355` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.160141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1183cb7ade7b4758b9e6a106b7980355)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.162057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.206660 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1183cb7ade7b4758b9e6a106b7980355'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.207316 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1183cb7ade7b4758b9e6a106b7980355'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.214373 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1183cb7ade7b4758b9e6a106b7980355'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.221352 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1183cb7ade7b4758b9e6a106b7980355'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.234811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1183cb7ade7b4758b9e6a106b7980355'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.240971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1360/2681] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:25 2022] DELETE /identity/v3/users/1183cb7ade7b4758b9e6a106b7980355 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:25.246758 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.247234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.247349 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.266565 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:25.267959 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.327968 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1322/2682] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:58:25.344400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.345835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.373924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:25.405817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f592282-e0a0-4a4a-b5b1-01c661f8b15d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.409403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.410007 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.410287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.410559 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.420034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.421464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab14b48-4369-4838-9cf1-733e26296fe2 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.431330 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1361/2683] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:25.706063 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.708122 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.739679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f836720-5148-419f-876c-fe04cfc370c5 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.747830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.748650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.749007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.749369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] PATH_INFO: `/v3/projects/c3f39580ab584e8ab9aa3a4d9e86d5d9` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.750792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f39580ab584e8ab9aa3a4d9e86d5d9)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.753111 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.802831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f39580ab584e8ab9aa3a4d9e86d5d9', 'request_id': 'req-fd018c32-666d-4e48-b9c9-47cb944d5cc2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.808437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f39580ab584e8ab9aa3a4d9e86d5d9', 'request_id': 'req-fd018c32-666d-4e48-b9c9-47cb944d5cc2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.815584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f39580ab584e8ab9aa3a4d9e86d5d9', 'request_id': 'req-fd018c32-666d-4e48-b9c9-47cb944d5cc2'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.827342 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] The token cache is being invalidate because project c3f39580ab584e8ab9aa3a4d9e86d5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:25.827727 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.828364 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fd018c32-666d-4e48-b9c9-47cb944d5cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:25.829891 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1323/2684] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:25 2022] DELETE /identity/v3/projects/c3f39580ab584e8ab9aa3a4d9e86d5d9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:25.835005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.835360 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.835641 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.855460 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:25.856716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.918254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1362/2685] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:25.926266 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.926490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.926764 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.947090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:25.948483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:25.957504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1324/2686] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:25.960965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:25.971401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c87ea-a7fa-432f-aa54-507e99194860 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:25.975996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:25.976587 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:25.976863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:25.977135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:25.978379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:25.978674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:25.980278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:25.991861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1363/2687] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:25.995977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.005223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196f034-d629-4fc6-9dc6-df8c3e87eb46 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.011819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.012422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.012700 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.012979 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.013584 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.015501 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.042952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1325/2688] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:25 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:26.047035 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.057038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4efe8-5f10-412c-893f-2ec810c1336c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.113747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.114338 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.114607 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.114873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.115571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.117528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.138930 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1364/2689] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:26 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:26.143078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.153853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88f6d-0e2b-4299-86eb-ec0e1880f452 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.158340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.158922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.159194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.159458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.160212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.161783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.171498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1326/2690] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:26.175300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.185319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29602176-9872-449e-ad1a-bbd57dbe4b2f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.189822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.190403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.190677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.190945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.191606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.193452 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251f326-7f50-46ba-8eba-19df71b53650 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.204547 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1365/2691] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:26.208300 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.218775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa442f-73c0-478f-8572-cd66e476fbbf admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.223531 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.224181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.224458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.224730 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] PATH_INFO: `/v3/projects/c2fb2694dbac4f9c8cdb2a6afb1cda97/users/fe9c7c215aa14c2a86c48da9e5f74846/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.239669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fb2694dbac4f9c8cdb2a6afb1cda97, user_id=fe9c7c215aa14c2a86c48da9e5f74846, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.242323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.257684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1327/2692] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:26 2022] PUT /identity/v3/projects/c2fb2694dbac4f9c8cdb2a6afb1cda97/users/fe9c7c215aa14c2a86c48da9e5f74846/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:26.268207 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.268494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.268765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.297915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] MFA Rules not processed for user `fe9c7c215aa14c2a86c48da9e5f74846`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:26.299126 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:26.317272 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:26.362698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1366/2693] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:26.367358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.376941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:26.433185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c6fa8-d61d-4b23-b56b-6e6242d7d088 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.438277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.438821 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.439099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.439368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.440027 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.441932 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.469498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1328/2694] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:26 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:26.473620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.485128 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169ce39-0c15-4ff8-a6ff-07077e18abbb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.544174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.544789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.545076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.545366 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.546006 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.547969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.568864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1367/2695] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:26 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:26.572961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.583064 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4b56-f444-463a-806b-d699e81fe822 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.588208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.588803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.589082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.589352 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.589985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.591636 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.601897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1329/2696] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:26.606430 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.616401 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da78ba-f666-492b-a390-662119b3bd50 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.621014 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.621681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.621961 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.622232 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] PATH_INFO: `/v3/projects/d174465c632b49d9acec89f999c44bf6/users/d375bbf123e04da0ab105d814d998991/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.636744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d174465c632b49d9acec89f999c44bf6, user_id=d375bbf123e04da0ab105d814d998991, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.639411 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.652735 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1368/2697] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:26 2022] PUT /identity/v3/projects/d174465c632b49d9acec89f999c44bf6/users/d375bbf123e04da0ab105d814d998991/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:26.656671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.667265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee1511-b941-4a3c-8733-4534e220a29b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.721750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.722360 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.722642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.722918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.723578 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.725167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c859e48-2a0c-4156-b3c8-da9526c68176 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.735208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1330/2698] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:26.739283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.749613 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a308-cdba-47a4-82d5-d9bfcfe81124 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.754222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.754864 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.755146 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.755419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] PATH_INFO: `/v3/projects/d174465c632b49d9acec89f999c44bf6/users/d375bbf123e04da0ab105d814d998991/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.757391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d174465c632b49d9acec89f999c44bf6, user_id=d375bbf123e04da0ab105d814d998991, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.760581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.770642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1369/2699] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:26 2022] PUT /identity/v3/projects/d174465c632b49d9acec89f999c44bf6/users/d375bbf123e04da0ab105d814d998991/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:26.782678 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.782970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.783241 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.810807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] MFA Rules not processed for user `d375bbf123e04da0ab105d814d998991`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:26.812168 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:26.887944 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1331/2700] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:26.899498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:26.901274 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:26.957860 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b56fb-b2b6-4152-99b7-d0cba4ede0c5 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:26.961574 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:26.962237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:26.962518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:26.962803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:26.973349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:26.974728 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:26.984833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1370/2701] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:27.059559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:27.064287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:27.071588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262889c-5111-457f-a60c-d9a4af4670a7 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:27.075499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:27.076095 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:27.076367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:27.076634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:27.134934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:27.136412 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:27.145406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1332/2702] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:29.020566 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:29.022666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:29.042993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:29.113791 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8685-b5df-4180-8129-a2af31bc1523 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:29.121845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:29.122158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:29.122526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:29.122887 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:29.124884 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:29.140710 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:29.142725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfc83c-39b1-48ca-9f11-1beadb96eb30 service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:29.171666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1371/2703] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:58:36.046651 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:36.109963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc382d7-fe1f-4583-9a60-1c45d4f67322 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:36.114573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:36.115208 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.115496 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.115789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] PATH_INFO: `/v3/users/fe9c7c215aa14c2a86c48da9e5f74846` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.116941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9c7c215aa14c2a86c48da9e5f74846)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:36.119720 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:36.166302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9c7c215aa14c2a86c48da9e5f74846'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.166869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9c7c215aa14c2a86c48da9e5f74846'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.173947 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9c7c215aa14c2a86c48da9e5f74846'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.186051 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9c7c215aa14c2a86c48da9e5f74846'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.193003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9c7c215aa14c2a86c48da9e5f74846'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.199395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1333/2704] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:36 2022] DELETE /identity/v3/users/fe9c7c215aa14c2a86c48da9e5f74846 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:36.203817 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.204299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.204400 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.223853 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:36.225063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:36.283064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1372/2705] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:36.292773 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:36.294170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:36.316222 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:36.353558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a715-61ea-4257-823c-c2d3365b7a28 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:36.357249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:36.357851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.358131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.358403 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.367279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:36.369253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:36.378781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1334/2706] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:36.671144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:36.672637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:36.689609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646d9ed-655f-4e55-bdf2-d8d1252838b6 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:36.695415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:36.696446 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.696803 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.697747 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] PATH_INFO: `/v3/projects/c2fb2694dbac4f9c8cdb2a6afb1cda97` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.698993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2fb2694dbac4f9c8cdb2a6afb1cda97)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:36.701022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:36.752193 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2fb2694dbac4f9c8cdb2a6afb1cda97', 'request_id': 'req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.758550 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2fb2694dbac4f9c8cdb2a6afb1cda97', 'request_id': 'req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.768518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2fb2694dbac4f9c8cdb2a6afb1cda97', 'request_id': 'req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.780905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] The token cache is being invalidate because project c2fb2694dbac4f9c8cdb2a6afb1cda97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:36.781240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.782179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.784057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1373/2707] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:36 2022] DELETE /identity/v3/projects/c2fb2694dbac4f9c8cdb2a6afb1cda97 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:36.787742 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:36.797795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179758a7-55d5-4970-b70f-827717bb9ad7 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:36.872749 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:36.873423 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.873704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.874139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] PATH_INFO: `/v3/users/d375bbf123e04da0ab105d814d998991` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.875186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:36.876614 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:36.876746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d375bbf123e04da0ab105d814d998991)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:36.878698 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:36.906149 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:36.927317 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd375bbf123e04da0ab105d814d998991'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.928005 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd375bbf123e04da0ab105d814d998991'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.936917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd375bbf123e04da0ab105d814d998991'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.939531 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1eb33-ecb9-48d3-83fb-10f004f7ab83 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:36.943370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:36.944012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:36.944307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:36.944581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:36.946357 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:36.950600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd375bbf123e04da0ab105d814d998991'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.957495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd375bbf123e04da0ab105d814d998991'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:36.958675 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:36.960124 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a745a-9a76-4e3d-b081-f835d0dff9be service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:36.964461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1335/2708] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:36 2022] DELETE /identity/v3/users/d375bbf123e04da0ab105d814d998991 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:36.973815 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1374/2709] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:58:37.241062 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.242620 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:37.295100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:37.316193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea297ed-dfc6-4c1b-8408-af439778be3a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.321115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.321751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.322079 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.322333 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] PATH_INFO: `/v3/projects/d174465c632b49d9acec89f999c44bf6` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.323499 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d174465c632b49d9acec89f999c44bf6)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.325422 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.374090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd174465c632b49d9acec89f999c44bf6', 'request_id': 'req-5b332676-4385-481b-b0b2-f304cb27b355'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:37.380206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd174465c632b49d9acec89f999c44bf6', 'request_id': 'req-5b332676-4385-481b-b0b2-f304cb27b355'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:37.387703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd174465c632b49d9acec89f999c44bf6', 'request_id': 'req-5b332676-4385-481b-b0b2-f304cb27b355'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:37.399984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] The token cache is being invalidate because project d174465c632b49d9acec89f999c44bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:37.400273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:37.400943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-5b332676-4385-481b-b0b2-f304cb27b355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:37.402738 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1336/2710] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:37 2022] DELETE /identity/v3/projects/d174465c632b49d9acec89f999c44bf6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:37.408266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.408771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.408873 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.429437 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:37.430664 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:37.493061 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1375/2711] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:37.502955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.503245 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.503559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.526551 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:37.532013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:37.535960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1337/2712] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:37.541203 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.550764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21eea97-8067-4a30-9056-783cdb382921 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.555364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.555998 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.556329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.557096 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.557740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:37.558039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.559652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.571385 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1376/2713] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:58:37.576308 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.585527 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4bfe4-e3af-4b5a-83c6-1457e073b9a9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.590774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.591753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.591753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.592277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.593380 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.594916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.625232 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1338/2714] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:37.629518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.640179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8884d63-d1e5-4d1c-9149-ac524a23c02d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.698382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.699074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.699351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.699650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.700440 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.702375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.727011 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1377/2715] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:37.728518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.738846 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163816a3-9fd4-4cd0-831a-2bdc931a45cd admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.744254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.744872 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.745156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.745430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.746068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.747688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.757740 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1339/2716] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:37.762296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.773153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2046-81e1-4541-a29a-a6e8de7b4175 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.778725 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.779077 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.779359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.779683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.780320 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.781921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beddbc8-0468-44e7-adb6-266a89fa5814 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.792784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1378/2717] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:37.796910 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:37.807574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a68f2-b3c6-4927-873f-6f6d616ca05e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:37.812291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:37.812942 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.813221 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.813495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] PATH_INFO: `/v3/projects/598126956aad4e06a79f4a2b253a278b/users/4cdb8cdf4c594f77a6658ae9303bc77d/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.828913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=598126956aad4e06a79f4a2b253a278b, user_id=4cdb8cdf4c594f77a6658ae9303bc77d, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:37.831796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:37.844532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1340/2718] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:37 2022] PUT /identity/v3/projects/598126956aad4e06a79f4a2b253a278b/users/4cdb8cdf4c594f77a6658ae9303bc77d/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:37.856189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:37.856523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:37.856752 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:37.889552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] MFA Rules not processed for user `4cdb8cdf4c594f77a6658ae9303bc77d`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:37.890836 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:37.915597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:37.970985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1379/2719] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:37.975591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.057351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42d8f-a991-4bf7-a836-4e27ee4e40c1 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.062086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.062699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.062984 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.063766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.064404 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.066313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.081124 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:38.096901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1341/2720] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:37 2022] POST /identity/v3/projects => generated 393 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:38.100686 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.112955 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fd4a0-f2c3-44de-a3bd-1346d7838906 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.182166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.182867 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.183217 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.183616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.184324 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.186314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.209632 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1380/2721] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:38 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:38.214416 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.225116 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15941e30-ea9c-4d52-ab74-0639dad78408 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.229941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.230546 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.230827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.231100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.231801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.233477 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.246283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1342/2722] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:38.250564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.261098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cc7c8-11d2-4277-a006-6211f9eb8822 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.266231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.266885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.267162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.267508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] PATH_INFO: `/v3/projects/9e0f0caa970641e4ae981522d2f90090/users/299dac5ace664273b3ffc088db10bacb/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.283086 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0f0caa970641e4ae981522d2f90090, user_id=299dac5ace664273b3ffc088db10bacb, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.285811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.298919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1381/2723] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:38 2022] PUT /identity/v3/projects/9e0f0caa970641e4ae981522d2f90090/users/299dac5ace664273b3ffc088db10bacb/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:38.302844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.312751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e6746-8bde-41a0-b877-0f765e617c09 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.378088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.378782 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.379131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.379472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.380213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.381918 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a83db-49d8-4903-8f43-4a6a02fe1aa7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.396227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1343/2724] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:38.397466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.409597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582ba3-4af1-420d-bf4e-3fc464a6d521 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.415688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.416525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.417082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.417608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] PATH_INFO: `/v3/projects/9e0f0caa970641e4ae981522d2f90090/users/299dac5ace664273b3ffc088db10bacb/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.419709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0f0caa970641e4ae981522d2f90090, user_id=299dac5ace664273b3ffc088db10bacb, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.423395 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.439300 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1382/2725] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:38 2022] PUT /identity/v3/projects/9e0f0caa970641e4ae981522d2f90090/users/299dac5ace664273b3ffc088db10bacb/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:38.506103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.506507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.506788 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.548535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] MFA Rules not processed for user `299dac5ace664273b3ffc088db10bacb`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:38.549792 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:38.632373 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1344/2726] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:38.640556 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.642374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:38.709043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d6de-80db-48aa-a895-ced782ca17a6 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.713133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.713833 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.714178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.714549 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.725737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.727298 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.738512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1383/2727] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:38 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:38.789137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:38.791732 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:38.804438 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663db7-723c-4d13-b3bc-1d69c436c1f8 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:38.809382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:38.810002 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:38.810420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:38.810711 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:38.876746 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:38.878453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819350b-88d7-46f9-9d10-57dc9f95c9c0 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:38.889682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1345/2728] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:42.373419 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:42.394914 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:42.457076 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfb623-c36e-4b1e-9d66-7bb5651b38c9 service cinder] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:42.462015 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:42.462708 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:42.462991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:42.463264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] PATH_INFO: `/v3/users/4cdb8cdf4c594f77a6658ae9303bc77d` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:42.464470 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cdb8cdf4c594f77a6658ae9303bc77d)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:42.466474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:42.517637 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cdb8cdf4c594f77a6658ae9303bc77d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:42.519286 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cdb8cdf4c594f77a6658ae9303bc77d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:42.529730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cdb8cdf4c594f77a6658ae9303bc77d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:42.542045 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cdb8cdf4c594f77a6658ae9303bc77d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:42.556538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cdb8cdf4c594f77a6658ae9303bc77d'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:42.570155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1384/2729] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:42 2022] DELETE /identity/v3/users/4cdb8cdf4c594f77a6658ae9303bc77d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:42.576076 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:42.576591 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:42.576694 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:42.584012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:42.599812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:42.601219 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:42.690115 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1346/2730] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:42.713565 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:42.715098 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:42.788351 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123a3c-a9dc-4081-9e36-0d44eacab80b admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:42.792689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:42.793920 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:42.794205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:42.794479 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:42.806516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:42.808161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:42.818971 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1385/2731] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:43.552940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:43.555712 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:43.565730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:43.568245 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:43.586573 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81648a-40d8-4c26-96d1-83a90e772933 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:43.594392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:43.595451 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:43.595772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:43.596154 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] PATH_INFO: `/v3/projects/598126956aad4e06a79f4a2b253a278b` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:43.597441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] RBAC: Authorizing `identity:delete_project(project_id=598126956aad4e06a79f4a2b253a278b)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:43.597916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1fa5-8580-4b41-8a13-0452c433b8db service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:43.601383 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:43.603806 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:43.604538 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:43.604880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:43.605601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] PATH_INFO: `/v3/users/4279761de1464be38860432a46a89074` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:43.606754 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4279761de1464be38860432a46a89074)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:43.609044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:43.659097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:43.670707 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598126956aad4e06a79f4a2b253a278b', 'request_id': 'req-6a8d07fd-c615-497f-9697-d4f7cc80716a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.674749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4279761de1464be38860432a46a89074'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.675666 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4279761de1464be38860432a46a89074'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.678981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598126956aad4e06a79f4a2b253a278b', 'request_id': 'req-6a8d07fd-c615-497f-9697-d4f7cc80716a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.685670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4279761de1464be38860432a46a89074'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.688808 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598126956aad4e06a79f4a2b253a278b', 'request_id': 'req-6a8d07fd-c615-497f-9697-d4f7cc80716a'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.693751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4279761de1464be38860432a46a89074'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.707417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4279761de1464be38860432a46a89074'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.708610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] The token cache is being invalidate because project 598126956aad4e06a79f4a2b253a278b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:43.708903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.709751 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:43.711461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1347/2732] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:43 2022] DELETE /identity/v3/projects/598126956aad4e06a79f4a2b253a278b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:43.714422 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1386/2733] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:43 2022] DELETE /identity/v3/users/4279761de1464be38860432a46a89074 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:43.716505 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:43.718436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:43.825491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d07fd-c615-497f-9697-d4f7cc80716a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:43.891094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:43.956898 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:43.958150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:43.958684 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:43.959340 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] PATH_INFO: `/v3/users/299dac5ace664273b3ffc088db10bacb` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:43.961088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299dac5ace664273b3ffc088db10bacb)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:43.965543 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.039639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299dac5ace664273b3ffc088db10bacb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.040440 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299dac5ace664273b3ffc088db10bacb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.050173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299dac5ace664273b3ffc088db10bacb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.065182 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299dac5ace664273b3ffc088db10bacb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.073961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299dac5ace664273b3ffc088db10bacb'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.080867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1348/2734] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:43 2022] DELETE /identity/v3/users/299dac5ace664273b3ffc088db10bacb => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:44.249093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.250606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:44.278056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27161-930a-4399-94af-f66b0ebb99b0 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:44.374163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:44.375083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.377384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.377898 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] PATH_INFO: `/v3/projects/b9cd3cd0a9434f5ab0e1e1c52145beb2` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.380168 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9cd3cd0a9434f5ab0e1e1c52145beb2)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:44.385196 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.445405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9cd3cd0a9434f5ab0e1e1c52145beb2', 'request_id': 'req-94a1454f-77c6-4cd4-b682-26c07a2bf186'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.453144 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9cd3cd0a9434f5ab0e1e1c52145beb2', 'request_id': 'req-94a1454f-77c6-4cd4-b682-26c07a2bf186'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.459139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.460690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:44.468075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9cd3cd0a9434f5ab0e1e1c52145beb2', 'request_id': 'req-94a1454f-77c6-4cd4-b682-26c07a2bf186'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.474669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2127544-c65c-43d1-882d-8eb19a3701c2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:44.481744 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] The token cache is being invalidate because project b9cd3cd0a9434f5ab0e1e1c52145beb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:44.484692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.484692 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.488103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1387/2735] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:44 2022] DELETE /identity/v3/projects/b9cd3cd0a9434f5ab0e1e1c52145beb2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:44.489141 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.490564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:44.572772 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:44.575957 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:44.577041 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.577491 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.577972 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] PATH_INFO: `/v3/projects/9e0f0caa970641e4ae981522d2f90090` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.579574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0f0caa970641e4ae981522d2f90090)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:44.582319 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.600562 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1454f-77c6-4cd4-b682-26c07a2bf186 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:44.608231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:44.609233 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.609690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.610132 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] PATH_INFO: `/v3/users/9d3e53bec1ed45a7a3a24cdd99c2dc01` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.612074 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3e53bec1ed45a7a3a24cdd99c2dc01)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:44.614655 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.645218 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0f0caa970641e4ae981522d2f90090', 'request_id': 'req-e28ce122-5d49-4368-9af8-ef476883a0f9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.652152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:44.656671 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0f0caa970641e4ae981522d2f90090', 'request_id': 'req-e28ce122-5d49-4368-9af8-ef476883a0f9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.667997 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0f0caa970641e4ae981522d2f90090', 'request_id': 'req-e28ce122-5d49-4368-9af8-ef476883a0f9'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.680916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3e53bec1ed45a7a3a24cdd99c2dc01'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.681800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3e53bec1ed45a7a3a24cdd99c2dc01'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.691209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3e53bec1ed45a7a3a24cdd99c2dc01'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.696086 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] The token cache is being invalidate because project 9e0f0caa970641e4ae981522d2f90090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:44.696628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.697463 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e28ce122-5d49-4368-9af8-ef476883a0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.704439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1349/2736] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:44 2022] DELETE /identity/v3/projects/9e0f0caa970641e4ae981522d2f90090 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:44.706096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.707908 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.708137 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3e53bec1ed45a7a3a24cdd99c2dc01'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.708632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.725464 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ed4f0add-d39c-4524-892e-ff627753a307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3e53bec1ed45a7a3a24cdd99c2dc01'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:44.734270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1388/2737] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:58:44 2022] DELETE /identity/v3/users/9d3e53bec1ed45a7a3a24cdd99c2dc01 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:44.737424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:44.738831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:44.817054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1350/2738] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:44.820267 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.820571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.820846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.854514 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:44.856680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:44.867683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1389/2739] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:44.871681 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.886254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f7ea6-81e8-4a94-9493-e211a313a976 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:44.893627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:44.895164 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.896367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.896817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.897672 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:58:44.898180 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:44.902038 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.921069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1351/2740] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 17:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:58:44.925533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.938929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290b48e-8782-4a67-8284-bc07b22a0b26 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:44.945553 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:44.948449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:44.948449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:44.948449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:44.948449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:44.950599 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:44.974468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:44.976650 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:45.006039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1390/2741] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 17:58:44 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:58:45.010261 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:45.018867 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c18b4-a14c-44c9-8919-25d83608347d admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.093498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84e6a-eda3-46a4-b193-80780f33c5cf admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.100219 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.100965 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.101313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.101658 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.102361 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.104448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.121385 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.122542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.124574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.124951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] PATH_INFO: `/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.126382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5ab9fdc1b0422fbd49556f5c55f8a5)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.128472 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.136056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1391/2742] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 17:58:45 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:58:45.137348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:45.148182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006f608-de8e-4fde-9b64-197056b6ded9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.155039 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.155917 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.156580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.156580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.157669 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.159432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.174109 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1392/2743] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:45.178459 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:45.195145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe602f65-6d9e-41c2-ba02-a621e52dc523 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.198569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5ab9fdc1b0422fbd49556f5c55f8a5', 'request_id': 'req-e0e32874-d228-4b70-9a4a-f97a8dd162f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:45.204768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:45.206043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5ab9fdc1b0422fbd49556f5c55f8a5', 'request_id': 'req-e0e32874-d228-4b70-9a4a-f97a8dd162f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:45.215830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5ab9fdc1b0422fbd49556f5c55f8a5', 'request_id': 'req-e0e32874-d228-4b70-9a4a-f97a8dd162f0'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:45.227355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:45.240439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] The token cache is being invalidate because project 1a5ab9fdc1b0422fbd49556f5c55f8a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:58:45.241190 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:45.242097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:58:45.243879 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1352/2744] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:58:44 2022] DELETE /identity/v3/projects/1a5ab9fdc1b0422fbd49556f5c55f8a5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:58:45.306205 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.306921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.307353 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.307677 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.308331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.309983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8c3f7-6643-4737-9968-9d389a3ff5d7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.323425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1393/2745] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 17:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:58:45.328605 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:45.333145 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:45.371504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e32874-d228-4b70-9a4a-f97a8dd162f0 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.379509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.379509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.379509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.379509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] PATH_INFO: `/v3/projects/d62cbdfea83843ed92d1d42055e24ed3/users/499ecab70f4c4c1dbf0be673be5eb52c/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.403861 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62cbdfea83843ed92d1d42055e24ed3, user_id=499ecab70f4c4c1dbf0be673be5eb52c, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.407250 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.431950 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1353/2746] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 17:58:45 2022] PUT /identity/v3/projects/d62cbdfea83843ed92d1d42055e24ed3/users/499ecab70f4c4c1dbf0be673be5eb52c/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:58:45.442064 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd13b-4052-4b3d-a3c9-171b59155bee None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.442916 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd13b-4052-4b3d-a3c9-171b59155bee None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.443349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd13b-4052-4b3d-a3c9-171b59155bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.503905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-55bdd13b-4052-4b3d-a3c9-171b59155bee None None] MFA Rules not processed for user `499ecab70f4c4c1dbf0be673be5eb52c`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:45.505846 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-55bdd13b-4052-4b3d-a3c9-171b59155bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:45.598441 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1394/2747] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:45.610127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:45.611886 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:45.676280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec444e70-8c6b-4188-a9e8-8511a5362d94 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:45.680040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:45.680679 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:45.681139 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:45.681411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:45.691066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:45.692866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc74dc1-5feb-44b2-9582-0c1efab933d0 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:45.702627 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1354/2748] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:46.709090 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:46.709601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:46.709722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:46.729659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] MFA Rules not processed for user `1daca2cfac58487189f1fe8bf4bcb130`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:46.730879 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:46.784311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1395/2749] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:47.084312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:47.084831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:47.084831 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:47.093387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:47.094996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.104952 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] MFA Rules not processed for user `499ecab70f4c4c1dbf0be673be5eb52c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:58:47.106151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.113355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1355/2750] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:58:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:58:47.145811 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:47.158020 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b5505-c590-47cc-8c2a-ee2aa0e0d348 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:47.161716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:47.162328 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:47.162608 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:47.163189 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:47.164826 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.175080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:47.176539 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:47.185363 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1396/2751] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:58:47.514995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:47.519312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.604059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:47.605469 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.606012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6e692-ea84-41af-8941-e16ce35b436b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:47.611723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:47.612328 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:47.612600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:47.612866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:47.614590 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:47.620462 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253fa98-8cec-45d9-92c9-1f076e0753f6 service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:47.625580 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:47.626389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:47.626679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:47.626958 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:47.627088 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:47.630092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:47.636342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:47.637935 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:47.639464 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1356/2752] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:47.649041 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1397/2753] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:50.394592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:50.397231 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:50.476943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:58:50.494483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81c19a-a90c-4e7e-93ec-d838b39bea41 service placement] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:50.501494 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:50.502407 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:50.502786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:50.503201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:50.505167 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:50.523723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:50.526025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fac1-08c9-4525-aa84-d52e5e64f475 service nova] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:50.540068 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1357/2754] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:58:54.670688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:58:54.685337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403effe1-edc9-42d0-8b81-64a5dd5e0b8d service placement] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:58:54.692821 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a6e9e58daeb47abb6277e7c2bf6bdcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:58:54.693585 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:58:54.693968 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:58:54.694347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:58:54.696313 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:58:54.710509 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:58:54.712337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:58:54.725386 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1398/2755] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:58:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:00.534226 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=6 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:01.278091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec2c2-4cdb-4d79-be61-e812697acff7 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:01.278375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec2c2-4cdb-4d79-be61-e812697acff7 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:01.278644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fec2c2-4cdb-4d79-be61-e812697acff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:01.338614 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-51fec2c2-4cdb-4d79-be61-e812697acff7 None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:59:01.339900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-51fec2c2-4cdb-4d79-be61-e812697acff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:01.402637 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1358/2756] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:59:01.448321 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:01.449738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:01.537349 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdede01-be20-4364-96d4-c114bc9be28a service nova] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:01.542891 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:01.543511 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:01.543784 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:01.544053 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:01.551937 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:01.553503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:01.561933 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1399/2757] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:59:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:59:02.049666 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee16c2-9d55-40c5-9294-bb628c7e0c3a None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.050134 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee16c2-9d55-40c5-9294-bb628c7e0c3a None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.050233 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee16c2-9d55-40c5-9294-bb628c7e0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.070069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a2ee16c2-9d55-40c5-9294-bb628c7e0c3a None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:59:02.071312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a2ee16c2-9d55-40c5-9294-bb628c7e0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:02.078572 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1359/2758] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:59:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:59:02.090564 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:02.102183 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024bf1d-e6cd-464b-b411-01549de772f8 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:02.107266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:02.107893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.108175 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.108447 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.116382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:02.118569 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:02.126773 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1400/2759] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:59:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:59:02.338659 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5355c5-154e-46de-9e1d-91ce5cbbca3b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.338937 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5355c5-154e-46de-9e1d-91ce5cbbca3b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.339207 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5355c5-154e-46de-9e1d-91ce5cbbca3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.358152 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-9e5355c5-154e-46de-9e1d-91ce5cbbca3b None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:59:02.359536 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9e5355c5-154e-46de-9e1d-91ce5cbbca3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:02.366516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1360/2760] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Aug 24 17:59:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:59:02.378439 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:02.388881 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d4a26-d9c9-479d-a3bb-054ffd950564 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:02.393888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:02.394477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.394753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.395023 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.403449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:02.405066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:02.413418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1401/2761] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:59:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:59:02.731379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.731690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.731990 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.751629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] MFA Rules not processed for user `c78ca59d6dbd409ea40d1a9991c8263c`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:59:02.752845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:02.760842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1361/2762] 10.222.0.27 () {58 vars in 967 bytes} [Wed Aug 24 17:59:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:59:02.772774 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:02.784111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d8115-7c95-4b6d-87f5-975f172b4785 service swift] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:02.789084 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee2ab73d700247e5b5bd72ee331bb4fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:02.789673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:02.789953 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:02.790223 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:02.798093 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:02.799682 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbab2d7-021a-4f54-a435-89fd7aadb111 service swift] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:02.807768 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1402/2763] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Aug 24 17:59:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:59:04.870080 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:04.931375 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65588ca5-ecc5-4e27-b697-9ff976e0d1aa None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:04.935977 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:04.936600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:04.936878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:04.937150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] PATH_INFO: `/v3/users/499ecab70f4c4c1dbf0be673be5eb52c` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:04.938286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499ecab70f4c4c1dbf0be673be5eb52c)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:04.940258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:04.984206 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:04.985748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499ecab70f4c4c1dbf0be673be5eb52c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:04.986519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499ecab70f4c4c1dbf0be673be5eb52c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:04.994981 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499ecab70f4c4c1dbf0be673be5eb52c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.007210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499ecab70f4c4c1dbf0be673be5eb52c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.014456 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499ecab70f4c4c1dbf0be673be5eb52c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.020335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1362/2764] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 17:59:04 2022] DELETE /identity/v3/users/499ecab70f4c4c1dbf0be673be5eb52c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:59:05.024753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:05.025236 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:05.025358 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:05.044079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:59:05.045271 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.088195 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:05.106005 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1403/2765] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 17:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:59:05.116500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:05.118682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.176270 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab145-8069-499a-a072-664ae4d16648 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:05.180479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:05.181078 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:05.181358 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:05.181628 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:05.190466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:05.192191 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:05.201592 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1363/2766] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:59:05.369753 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:59:05.371239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:59:05.383346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149a343-4a87-4cf9-a9b8-a637218584ba None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:59:05.387985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:59:05.388623 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:59:05.388902 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:59:05.389177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] PATH_INFO: `/v3/projects/d62cbdfea83843ed92d1d42055e24ed3` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:59:05.390332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62cbdfea83843ed92d1d42055e24ed3)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:59:05.392306 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:59:05.439952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62cbdfea83843ed92d1d42055e24ed3', 'request_id': 'req-ad8591c0-faff-45ad-b3d9-cb8631817776'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.445819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62cbdfea83843ed92d1d42055e24ed3', 'request_id': 'req-ad8591c0-faff-45ad-b3d9-cb8631817776'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.453234 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62cbdfea83843ed92d1d42055e24ed3', 'request_id': 'req-ad8591c0-faff-45ad-b3d9-cb8631817776'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.464974 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] The token cache is being invalidate because project d62cbdfea83843ed92d1d42055e24ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:59:05.465264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.465895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:59:05.467370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1404/2767] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 17:59:05 2022] DELETE /identity/v3/projects/d62cbdfea83843ed92d1d42055e24ed3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:59:15.197522 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=39,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:59:15.462001 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:05.008430 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.010550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.025516 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958d6e-4c1b-44f2-a6e2-ba1571a7da5b service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.087021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.087714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.087987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.088256 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] PATH_INFO: `/v3/users/1daca2cfac58487189f1fe8bf4bcb130` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.089512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1daca2cfac58487189f1fe8bf4bcb130)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.091439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.140392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1daca2cfac58487189f1fe8bf4bcb130'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.141212 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1daca2cfac58487189f1fe8bf4bcb130'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.148648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1daca2cfac58487189f1fe8bf4bcb130'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.160824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1daca2cfac58487189f1fe8bf4bcb130'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.168125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6b129dce-4cb3-45f5-ae79-9a0257163f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1daca2cfac58487189f1fe8bf4bcb130'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.173962 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1364/2768] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:05 2022] DELETE /identity/v3/users/1daca2cfac58487189f1fe8bf4bcb130 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:05.340764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.342283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.356346 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8591c0-faff-45ad-b3d9-cb8631817776 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.418631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.419173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.419449 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.419746 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] PATH_INFO: `/v3/projects/1422bbfed9414b8cada1656aed195a31` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.420854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1422bbfed9414b8cada1656aed195a31)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.422994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.469939 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1422bbfed9414b8cada1656aed195a31', 'request_id': 'req-92fe2e14-bc44-4347-8f9b-998a7085eac0'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.475403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1422bbfed9414b8cada1656aed195a31', 'request_id': 'req-92fe2e14-bc44-4347-8f9b-998a7085eac0'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.482145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1422bbfed9414b8cada1656aed195a31', 'request_id': 'req-92fe2e14-bc44-4347-8f9b-998a7085eac0'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.492832 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] The token cache is being invalidate because project 1422bbfed9414b8cada1656aed195a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:05.493114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.493730 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-92fe2e14-bc44-4347-8f9b-998a7085eac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:05.495179 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1405/2769] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:05 2022] DELETE /identity/v3/projects/1422bbfed9414b8cada1656aed195a31 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:05.500093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.500379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.500653 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.520151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:05.521348 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.579629 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1365/2770] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:05.587180 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.587468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.587770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.607643 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:05.608854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:05.617135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1406/2771] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:05.621202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.630676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba5439-7a3c-4b9b-b5cc-411e72962dbb None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.635205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790803f3-34f4-436f-972f-681270404443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.635913 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790803f3-34f4-436f-972f-681270404443 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.636188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790803f3-34f4-436f-972f-681270404443 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.636961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790803f3-34f4-436f-972f-681270404443 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.637588 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790803f3-34f4-436f-972f-681270404443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:05.637885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790803f3-34f4-436f-972f-681270404443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.639469 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790803f3-34f4-436f-972f-681270404443 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.650648 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1366/2772] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:05.655403 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.664882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20028df-5153-4423-a40d-0bb1fefcec2a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.669616 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.670147 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.670423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.670690 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.671304 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.673237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.700026 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1407/2773] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:05 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:05.704329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790803f3-34f4-436f-972f-681270404443 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.714156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790803f3-34f4-436f-972f-681270404443 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.718289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:05.774804 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.775410 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.775717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.775988 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.776618 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.778643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.798783 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1367/2774] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:05 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 18:00:05.803145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.813805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b091f-fb52-4d8c-83d8-58b06a647b84 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.818596 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.819182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.819456 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.819755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.820383 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.821954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.832239 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1408/2775] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:05.836125 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.846335 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad08785-98e0-4726-9c6f-047d69aca401 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.851105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.851726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.852006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.852277 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.852912 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.854495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2006a-94d3-4a62-b37a-4f32b4ee5dad admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.864779 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1368/2776] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:05.868716 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:05.879444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd76f21-92a1-45e5-a0a4-ca4f8ac46186 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:05.883992 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:05.884626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.884901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.885167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] PATH_INFO: `/v3/projects/aa7291fc029a453489c394e88d56a11e/users/c27b1749fc0b43d4aa26ba2e97138b14/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.900044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7291fc029a453489c394e88d56a11e, user_id=c27b1749fc0b43d4aa26ba2e97138b14, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:05.902777 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:05.915468 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1409/2777] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:05 2022] PUT /identity/v3/projects/aa7291fc029a453489c394e88d56a11e/users/c27b1749fc0b43d4aa26ba2e97138b14/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:05.926945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:05.927170 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:05.927634 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:05.956156 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] MFA Rules not processed for user `c27b1749fc0b43d4aa26ba2e97138b14`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:05.957365 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:06.021249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1369/2778] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:06.025824 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.054893 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:06.088800 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031da006-ba2f-4e39-ae92-9d91771402da admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.093475 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.094100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.094379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.094651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.095287 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.097210 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.125139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1410/2779] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:06 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:06.129015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.140113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0cc4b-d1e0-4721-bf6c-84465cd91b5e None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.203519 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.204057 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.204336 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.204609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.205229 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.207146 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.233312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1370/2780] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:06 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:06.234522 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.247454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aaa739-68b6-422a-815d-c2676952d3f3 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.252299 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.252973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.253254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.253523 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.254159 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.255787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.266827 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1411/2781] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:06.271435 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.278099 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:06.281845 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c05cb-2fb7-490a-9c1e-bb46de495f6b admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.287726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.288398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.288674 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.288943 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] PATH_INFO: `/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0/users/b1bd220983874eff97da9834c4150d79/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.304060 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=517fd4fb70a84d5db343d6ab64aaa3a0, user_id=b1bd220983874eff97da9834c4150d79, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.306717 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.320155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1371/2782] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:06 2022] PUT /identity/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0/users/b1bd220983874eff97da9834c4150d79/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:06.324121 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.334164 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bc414-9661-436c-8a4c-575f9193f93d admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.391314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.392290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.392290 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.392631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.393280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.394928 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfe236-340d-4f3a-be63-15cf4fea7552 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.404738 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1412/2783] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:06.408901 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.419428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137dcb-2646-457b-a22d-3bce8a6d5efa admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.424195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.424839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.425117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.425386 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] PATH_INFO: `/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0/users/b1bd220983874eff97da9834c4150d79/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.427289 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=517fd4fb70a84d5db343d6ab64aaa3a0, user_id=b1bd220983874eff97da9834c4150d79, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.429963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.440457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1372/2784] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:06 2022] PUT /identity/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0/users/b1bd220983874eff97da9834c4150d79/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:06.450796 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.451079 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.451348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.479331 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] MFA Rules not processed for user `b1bd220983874eff97da9834c4150d79`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:06.481199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:06.548162 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1413/2785] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:06.559526 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.561297 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:06.618390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cb164-1764-4be7-953a-39481b17e00f admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.622020 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.622609 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.622885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.623149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.631892 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.633305 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.642446 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1373/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:06.715256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:06.725263 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8383d4b-f291-4df2-acc1-d48f3fb5dd70 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:06.728862 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:06.729504 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:06.729729 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:06.729996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:06.749314 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:06.787604 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:06.789058 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db449e-0364-4868-a282-5524e23bc6ce service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:06.798652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1414/2787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:16.638379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:16.794552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=18 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:44.962688 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:45.024321 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752062d3-2163-41bd-b5b1-b6383c859603 service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:45.028907 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:45.029532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:45.029812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:45.030084 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] PATH_INFO: `/v3/users/c27b1749fc0b43d4aa26ba2e97138b14` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:45.031202 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27b1749fc0b43d4aa26ba2e97138b14)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:45.033217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:45.091200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27b1749fc0b43d4aa26ba2e97138b14'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.091842 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27b1749fc0b43d4aa26ba2e97138b14'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.100225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27b1749fc0b43d4aa26ba2e97138b14'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.115203 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27b1749fc0b43d4aa26ba2e97138b14'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.121945 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27b1749fc0b43d4aa26ba2e97138b14'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.127922 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1374/2788] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:44 2022] DELETE /identity/v3/users/c27b1749fc0b43d4aa26ba2e97138b14 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:45.132575 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:45.133063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:45.133176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:45.152111 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:45.153310 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:45.211819 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1415/2789] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:45.222264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:45.223697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:45.279996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e451a8c-9f07-4d6d-852e-d95950398038 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:45.283682 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:45.284228 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:45.284503 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:45.284772 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:45.293550 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:45.294978 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:45.304807 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1375/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:45.572679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:45.574142 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:45.586108 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb31188-5ed7-4c4f-b93b-cf0c7a25debb None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:45.590844 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:45.591499 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:45.591788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:45.592062 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] PATH_INFO: `/v3/projects/aa7291fc029a453489c394e88d56a11e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:45.593182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7291fc029a453489c394e88d56a11e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:45.595095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:45.641969 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7291fc029a453489c394e88d56a11e', 'request_id': 'req-16aabeed-d813-4d9a-aa10-f573c250745f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.647645 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7291fc029a453489c394e88d56a11e', 'request_id': 'req-16aabeed-d813-4d9a-aa10-f573c250745f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.654668 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7291fc029a453489c394e88d56a11e', 'request_id': 'req-16aabeed-d813-4d9a-aa10-f573c250745f'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.666025 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] The token cache is being invalidate because project aa7291fc029a453489c394e88d56a11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:45.666315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.666957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.668480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1416/2791] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:45 2022] DELETE /identity/v3/projects/aa7291fc029a453489c394e88d56a11e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:45.672639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:45.682824 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f33b-cd27-404a-ae22-d051f6e09ebe service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:45.701326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:45.740774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:45.741391 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:45.741669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:45.741939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] PATH_INFO: `/v3/users/b1bd220983874eff97da9834c4150d79` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:45.743054 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1bd220983874eff97da9834c4150d79)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:45.744996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:45.789439 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1bd220983874eff97da9834c4150d79'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.790082 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1bd220983874eff97da9834c4150d79'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.797007 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1bd220983874eff97da9834c4150d79'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.809753 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1bd220983874eff97da9834c4150d79'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.817390 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-51e7c10a-422e-4c44-ac1c-84af6f79a330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1bd220983874eff97da9834c4150d79'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:45.823421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1376/2792] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:45 2022] DELETE /identity/v3/users/b1bd220983874eff97da9834c4150d79 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:46.094840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.096571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:46.111863 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aabeed-d813-4d9a-aa10-f573c250745f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.168103 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.168781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.169022 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.169296 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] PATH_INFO: `/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.170420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=517fd4fb70a84d5db343d6ab64aaa3a0)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.172371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.202317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:46.219382 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517fd4fb70a84d5db343d6ab64aaa3a0', 'request_id': 'req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:46.225393 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517fd4fb70a84d5db343d6ab64aaa3a0', 'request_id': 'req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:46.232721 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517fd4fb70a84d5db343d6ab64aaa3a0', 'request_id': 'req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:46.244078 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] The token cache is being invalidate because project 517fd4fb70a84d5db343d6ab64aaa3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:46.244367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:46.245000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-24d52a0c-442c-4ca0-99ad-841ef6c8a1bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:46.246477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1417/2793] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:46 2022] DELETE /identity/v3/projects/517fd4fb70a84d5db343d6ab64aaa3a0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:46.251381 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.251897 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.252011 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.271073 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:46.272302 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:46.330504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1377/2794] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:46.338498 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.338723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.338995 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.364845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:46.366089 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:46.374256 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1418/2795] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:46.378561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.388287 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c7f6a-304d-4c75-bd18-1e55a4ebf579 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.392839 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.393441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.393716 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.393987 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.394617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:46.394915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.396534 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.408370 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1378/2796] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:46.413674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.423038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2718c-cbb0-4489-a833-38f74db6eff2 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.428372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.428973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.429252 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.429525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.430166 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.432102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.459731 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1419/2797] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:46.463896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.474561 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c610-4250-443b-992f-bdfa3d7f43e2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.500258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:46.532793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.534137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.534460 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.534767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.535437 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.537424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.559214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1379/2798] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 18:00:46.563341 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.574588 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6fdc-0fcc-4c59-be6b-c3914c4a7bc6 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.579238 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.579861 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.580148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.580418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.581055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.582647 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.592405 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1420/2799] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:46.596118 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.606029 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f21c5-9ceb-4241-98a2-c37dde722e17 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.610652 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.611771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.612045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.612314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.612939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.614506 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25804de0-a248-418d-8ac0-b84e0fd01232 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.624747 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1380/2800] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:46.629407 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.639512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4978a-6ef7-4171-8102-31e88a8fc1b5 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.644309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.644957 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.645237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.645507 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] PATH_INFO: `/v3/projects/0bdc687076f34e0aa3dfdfb7b1401f13/users/07913b17fdac4b06a7f9dab1bbe157c1/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.660443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdc687076f34e0aa3dfdfb7b1401f13, user_id=07913b17fdac4b06a7f9dab1bbe157c1, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.663105 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.676315 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1421/2801] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:46 2022] PUT /identity/v3/projects/0bdc687076f34e0aa3dfdfb7b1401f13/users/07913b17fdac4b06a7f9dab1bbe157c1/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:46.686750 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.687033 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.687304 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.715136 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] MFA Rules not processed for user `07913b17fdac4b06a7f9dab1bbe157c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:46.716372 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:46.779295 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1381/2802] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:46.783941 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.843688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de873704-b143-4f99-8c05-8cd74ee9bb4f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.848309 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.848906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.849182 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.849448 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.850057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.851973 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.878757 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1422/2803] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:46.882847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.893450 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9056a4-81df-4671-8f31-650b72b44a43 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:46.951311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:46.951946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:46.952224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:46.952495 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:46.953105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:46.955019 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:46.987817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1382/2804] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:46 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:46.989047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:46.999492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6578b-5899-4205-bdc1-7df7a6554338 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.004276 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.004878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.005156 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.005480 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.006115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.007732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.018057 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1423/2805] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:47.022243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.032279 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364833e5-5874-4fec-9abd-42224bf18ba6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.037356 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.038003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.038283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.038555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] PATH_INFO: `/v3/projects/326968c25e0d475a8c8b48cdaa0854ef/users/27ad75d162844e0cab63ab677f24d94a/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.061162 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=326968c25e0d475a8c8b48cdaa0854ef, user_id=27ad75d162844e0cab63ab677f24d94a, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.063882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.078347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1383/2806] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:47 2022] PUT /identity/v3/projects/326968c25e0d475a8c8b48cdaa0854ef/users/27ad75d162844e0cab63ab677f24d94a/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:47.082280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.092224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aeb09-c6e5-43d0-a5fc-8c9d720c1df9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.114938 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:47.151924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.152524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.152798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.153063 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.153681 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.155264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df94a98-8dbb-44bd-8339-bde10218e341 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.165342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1424/2807] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:47.169490 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.175610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:47.181869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c794a2-1ea2-4284-b4ee-c4e9b9d320b9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.186571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.187213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.187512 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.187780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] PATH_INFO: `/v3/projects/326968c25e0d475a8c8b48cdaa0854ef/users/27ad75d162844e0cab63ab677f24d94a/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.189690 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=326968c25e0d475a8c8b48cdaa0854ef, user_id=27ad75d162844e0cab63ab677f24d94a, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.192334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.202429 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1384/2808] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:47 2022] PUT /identity/v3/projects/326968c25e0d475a8c8b48cdaa0854ef/users/27ad75d162844e0cab63ab677f24d94a/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:47.213503 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.214009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.214112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.242097 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] MFA Rules not processed for user `27ad75d162844e0cab63ab677f24d94a`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:47.243302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:47.308978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1425/2809] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:47.321655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.323453 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:47.381436 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fa51d-566a-4ed7-9c3f-93ae6412b9cc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.384963 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.385558 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.385834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.386100 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.395458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.396916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.406265 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1385/2810] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:47.709412 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.718570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1950b7-3578-41da-b3b7-74f7d6b43879 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.723056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27ad75d162844e0cab63ab677f24d94a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '326968c25e0d475a8c8b48cdaa0854ef', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.723699 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.723981 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.724258 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.724895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:47.725197 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.726780 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.738051 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1426/2811] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:47.742021 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.750995 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ec62-4eb8-43b7-b033-333ecfe3ccae service cinder] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.756103 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27ad75d162844e0cab63ab677f24d94a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '326968c25e0d475a8c8b48cdaa0854ef', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.756687 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.756967 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.757242 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.757983 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.759919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.785834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1386/2812] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:47 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:47.789855 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.799620 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305af0-3a70-4e50-a1ac-eadc4fc70676 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.855802 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27ad75d162844e0cab63ab677f24d94a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '326968c25e0d475a8c8b48cdaa0854ef', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.856402 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.856706 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.856978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.857631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:47.857934 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.859524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.870765 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1427/2813] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 18:00:47.907691 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:47.920949 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af61f8-0434-492d-b2ee-8d424c100791 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:47.926045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27ad75d162844e0cab63ab677f24d94a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '326968c25e0d475a8c8b48cdaa0854ef', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:47.926663 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:47.926940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:47.927205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] PATH_INFO: `/v3/projects/27efb3ca95424f29b5332f1d3eec0e96` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:47.928347 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorizing `identity:delete_project(project_id=27efb3ca95424f29b5332f1d3eec0e96)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:47.930262 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:47.976276 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27efb3ca95424f29b5332f1d3eec0e96', 'request_id': 'req-acba52eb-c795-478f-b117-a031cedb2507'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:47.981704 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27efb3ca95424f29b5332f1d3eec0e96', 'request_id': 'req-acba52eb-c795-478f-b117-a031cedb2507'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:47.989487 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27efb3ca95424f29b5332f1d3eec0e96', 'request_id': 'req-acba52eb-c795-478f-b117-a031cedb2507'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.001040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] The token cache is being invalidate because project 27efb3ca95424f29b5332f1d3eec0e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:48.001326 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.001964 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-acba52eb-c795-478f-b117-a031cedb2507 tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.003445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1387/2814] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:47 2022] DELETE /identity/v3/projects/27efb3ca95424f29b5332f1d3eec0e96 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:48.190688 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:48.192421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:48.224139 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=100 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:48.260919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b92e2-0293-4fe3-8d3a-4a675cbc247f tempest-VolumeQuotaClassesTest-967379598 tempest-VolumeQuotaClassesTest-967379598-project-admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:48.314072 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:48.315266 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:48.315571 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:48.315845 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] PATH_INFO: `/v3/users/07913b17fdac4b06a7f9dab1bbe157c1` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:48.316985 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] RBAC: Authorizing `identity:delete_user(user_id=07913b17fdac4b06a7f9dab1bbe157c1)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:48.318911 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:48.366947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07913b17fdac4b06a7f9dab1bbe157c1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.367646 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07913b17fdac4b06a7f9dab1bbe157c1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.374894 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07913b17fdac4b06a7f9dab1bbe157c1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.382387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07913b17fdac4b06a7f9dab1bbe157c1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.394043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07913b17fdac4b06a7f9dab1bbe157c1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.399996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1428/2815] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:48 2022] DELETE /identity/v3/users/07913b17fdac4b06a7f9dab1bbe157c1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:48.404492 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:48.405013 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:48.405132 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:48.425251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:48.426394 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:48.472919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:48.486214 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1388/2816] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:48.496384 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:48.497792 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:48.554618 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e0e4-42b1-4ff8-b107-ba42bbf675de admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:48.558174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:48.558770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:48.559050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:48.559343 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:48.566153 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:48.568640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:48.570895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:48.581751 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1429/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:48.842090 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:48.843589 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:48.856960 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344f5d5-01ba-4fe2-88d8-94a42e633f14 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:48.861775 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:48.862940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:48.863225 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:48.863731 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] PATH_INFO: `/v3/projects/0bdc687076f34e0aa3dfdfb7b1401f13` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:48.865161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdc687076f34e0aa3dfdfb7b1401f13)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:48.867378 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:48.914819 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdc687076f34e0aa3dfdfb7b1401f13', 'request_id': 'req-7dd94f9c-99fa-4674-b216-b3fc0176362c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.920379 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdc687076f34e0aa3dfdfb7b1401f13', 'request_id': 'req-7dd94f9c-99fa-4674-b216-b3fc0176362c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.927878 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdc687076f34e0aa3dfdfb7b1401f13', 'request_id': 'req-7dd94f9c-99fa-4674-b216-b3fc0176362c'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.938803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] The token cache is being invalidate because project 0bdc687076f34e0aa3dfdfb7b1401f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:48.939093 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.939763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:48.941243 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1389/2818] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:48 2022] DELETE /identity/v3/projects/0bdc687076f34e0aa3dfdfb7b1401f13 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:48.945694 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:48.956028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25cb6a-8b79-47fc-8e99-be71f09eda5a service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.010770 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.011325 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.011642 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.011918 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] PATH_INFO: `/v3/users/27ad75d162844e0cab63ab677f24d94a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.013027 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ad75d162844e0cab63ab677f24d94a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.014986 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.066839 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ad75d162844e0cab63ab677f24d94a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.067506 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ad75d162844e0cab63ab677f24d94a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.075178 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ad75d162844e0cab63ab677f24d94a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.082032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ad75d162844e0cab63ab677f24d94a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.093579 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-5bff89a4-a087-4d93-a9e5-885091642a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ad75d162844e0cab63ab677f24d94a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.099302 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1430/2819] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:48 2022] DELETE /identity/v3/users/27ad75d162844e0cab63ab677f24d94a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:49.373961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.375400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:49.388362 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd94f9c-99fa-4674-b216-b3fc0176362c admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.445655 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.446314 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.446549 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.446827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] PATH_INFO: `/v3/projects/326968c25e0d475a8c8b48cdaa0854ef` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.447970 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326968c25e0d475a8c8b48cdaa0854ef)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.449874 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.496660 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326968c25e0d475a8c8b48cdaa0854ef', 'request_id': 'req-9dd78362-3613-4bb3-981e-8c98aaa3b463'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.502149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326968c25e0d475a8c8b48cdaa0854ef', 'request_id': 'req-9dd78362-3613-4bb3-981e-8c98aaa3b463'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.509685 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326968c25e0d475a8c8b48cdaa0854ef', 'request_id': 'req-9dd78362-3613-4bb3-981e-8c98aaa3b463'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.521261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] The token cache is being invalidate because project 326968c25e0d475a8c8b48cdaa0854ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:49.521559 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.522234 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-9dd78362-3613-4bb3-981e-8c98aaa3b463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:49.523780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1390/2820] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:49 2022] DELETE /identity/v3/projects/326968c25e0d475a8c8b48cdaa0854ef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:49.528705 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.529158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.529270 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.549711 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:49.550851 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:49.608329 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1431/2821] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:49.616803 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.617087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.617367 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.632173 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:49.638475 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:49.640570 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:49.649431 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1391/2822] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:49.653715 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.663994 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33cfeb-9017-4487-8506-e4213e49320a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.668640 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.669237 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.669518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.669789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.670425 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:49.670789 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.672443 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.683755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1432/2823] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:49.688040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.698004 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1695311-d763-4524-97ef-4cf7e1f19ec6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.702693 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.703285 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.703600 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.703873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.704486 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.706906 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.735396 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1392/2824] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:49 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:49.739494 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.744581 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:49.751042 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff935f8c-f44e-4a71-83b0-e8f4c1cba9e5 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.807529 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.808052 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.808339 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.808612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.809240 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.811155 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.832590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1433/2825] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:49 2022] POST /identity/v3/users => generated 281 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:49.836786 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.847066 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c5140-4e4f-4d68-961b-eee0979cb42e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.851692 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.852282 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.852557 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.852823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.853447 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.855188 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.865395 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1393/2826] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 18:00:49.869224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.880116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de58cf0-e8f2-49b2-96d4-5b13ded075e7 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.884696 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.885280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.885557 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.885830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.886495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.888115 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296c3eb-5181-4d8a-83cc-7c1cbf2b00c3 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.898317 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1434/2827] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:49.902311 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:49.912654 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcf394-4564-4054-9ec6-951ec34857ae admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:49.917197 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:49.917843 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.918119 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.918389 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] PATH_INFO: `/v3/projects/983d0e45a8b94786802f54d489c160a0/users/96061d91897f499fa64bbe9147ad2ea1/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.933574 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983d0e45a8b94786802f54d489c160a0, user_id=96061d91897f499fa64bbe9147ad2ea1, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:49.936258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:49.949521 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1394/2828] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:49 2022] PUT /identity/v3/projects/983d0e45a8b94786802f54d489c160a0/users/96061d91897f499fa64bbe9147ad2ea1/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:49.960450 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:49.960737 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:49.961012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:49.987636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] MFA Rules not processed for user `96061d91897f499fa64bbe9147ad2ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:49.988854 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.053167 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1435/2829] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:50.086406 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.088143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.146382 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728524c3-d6d3-4bf4-b3dd-e0e5531718b2 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.150018 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.150616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.150893 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.151160 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.159334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.160801 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088354-7dd0-455f-b32e-95721eb0a746 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:50.169616 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1395/2830] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:50.184991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.247073 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea833-dbdb-41bb-b14d-33c81e18897d None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.251683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.252830 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.253102 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.253370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] PATH_INFO: `/v3/users/96061d91897f499fa64bbe9147ad2ea1` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.254598 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] RBAC: Authorizing `identity:delete_user(user_id=96061d91897f499fa64bbe9147ad2ea1)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.256552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:50.268431 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:50.302500 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96061d91897f499fa64bbe9147ad2ea1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.303133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96061d91897f499fa64bbe9147ad2ea1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.310545 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96061d91897f499fa64bbe9147ad2ea1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.317943 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96061d91897f499fa64bbe9147ad2ea1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.329701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96061d91897f499fa64bbe9147ad2ea1'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.335540 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1436/2831] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:50 2022] DELETE /identity/v3/users/96061d91897f499fa64bbe9147ad2ea1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:50.340283 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.340569 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.340844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.359196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:50.360441 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.419946 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1396/2832] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:50.430896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.432311 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.489367 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232bf4b-39e2-439a-a8a7-ec45793c403c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.493010 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.493612 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.493889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.494161 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.503791 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.505225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ffa97-70fa-4239-8f77-8efa8be40889 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:50.514521 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1437/2833] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:50.774545 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:50.776102 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.788993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf48da-5eac-427d-9554-28dcd3664e66 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:50.793617 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:50.794334 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.794675 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.795418 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] PATH_INFO: `/v3/projects/983d0e45a8b94786802f54d489c160a0` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.796249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=983d0e45a8b94786802f54d489c160a0)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:50.798427 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:50.838936 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:50.846458 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983d0e45a8b94786802f54d489c160a0', 'request_id': 'req-a7c122c2-d0bb-4682-8c42-43ea8ef65531'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.852562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983d0e45a8b94786802f54d489c160a0', 'request_id': 'req-a7c122c2-d0bb-4682-8c42-43ea8ef65531'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.859931 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983d0e45a8b94786802f54d489c160a0', 'request_id': 'req-a7c122c2-d0bb-4682-8c42-43ea8ef65531'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.871359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] The token cache is being invalidate because project 983d0e45a8b94786802f54d489c160a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:50.871676 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.872323 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-a7c122c2-d0bb-4682-8c42-43ea8ef65531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:50.873796 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1397/2834] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:50 2022] DELETE /identity/v3/projects/983d0e45a8b94786802f54d489c160a0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:50.878704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:50.879163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:50.879264 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:50.898651 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:50.899905 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:50.959250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1438/2835] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:51.021778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.022143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.022420 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.042991 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:51.044419 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:51.053195 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1398/2836] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:51.057417 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.067134 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68843f54-0466-4363-9a3d-e5c805fcd582 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.072215 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.072818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.073095 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.073364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.074003 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:51.074307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.075945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.087763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1439/2837] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:51.091887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.101556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6ef1-1f2a-4ad9-a040-b229dc01a359 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.107077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.108181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.108181 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.109443 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.110189 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.112205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.138780 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1399/2838] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 18:00:51.143387 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.153389 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa21d70-0325-435c-a946-2c9d134ec026 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.170787 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:51.211733 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.212347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.212631 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.212906 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.213525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.215563 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.237635 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1440/2839] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:51.241870 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.252640 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878a79a-55d8-4164-b15b-cea6ad6ea744 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.257257 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.257851 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.258131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.258405 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.259040 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.260669 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.271351 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1400/2840] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.275277 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.285732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad86851-96f7-44a0-8230-8ea67076f3cc admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.290347 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.290940 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.291222 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.291512 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.292148 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.293740 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896da4b-e3c8-4c07-a979-e3721d4c189f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.304890 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1441/2841] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.308926 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.320161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa80ca-829c-4528-82bd-128d6b0b5fa9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.324881 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.325556 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.325827 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.326097 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] PATH_INFO: `/v3/projects/fb1293005af84968832027558519c91b/users/afaa5dbe9e98497392b7999515e9528e/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.340331 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1293005af84968832027558519c91b, user_id=afaa5dbe9e98497392b7999515e9528e, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.342961 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.355714 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1401/2842] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:51 2022] PUT /identity/v3/projects/fb1293005af84968832027558519c91b/users/afaa5dbe9e98497392b7999515e9528e/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:51.366174 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.366404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.366683 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.394055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] MFA Rules not processed for user `afaa5dbe9e98497392b7999515e9528e`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:51.395278 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:51.458135 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1442/2843] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:51.462778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.524205 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c525a2-c259-4e0c-a438-234a4b2007f9 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.529050 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.529658 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.529940 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.530213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.530840 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.532793 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.560210 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1402/2844] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:51.564269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.575380 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37cf1-8427-4124-85b3-2093a4504f55 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.630364 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.630895 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.631177 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.631444 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.632100 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.634004 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.656342 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1443/2845] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:51.660852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.671015 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22ba8-a19b-4766-9859-8c1e00a17b57 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.676230 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.676849 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.677127 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.677398 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.678026 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.679745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.690479 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1403/2846] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.694432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.705533 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19066ae-34e3-450d-9221-0c2f25e431f9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.710567 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.711213 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.711520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.711801 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] PATH_INFO: `/v3/projects/5f31299a7bd74105980480b171684977/users/50372d8baab24bfdba296b37b3fb4ce7/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.726170 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f31299a7bd74105980480b171684977, user_id=50372d8baab24bfdba296b37b3fb4ce7, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.728929 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.731101 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:51.743225 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1444/2847] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:51 2022] PUT /identity/v3/projects/5f31299a7bd74105980480b171684977/users/50372d8baab24bfdba296b37b3fb4ce7/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:00:51.747341 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.757951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8712-e385-4bc7-bc86-f1498b605ad6 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.794320 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:51.815192 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.815817 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.816126 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.816402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.817039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.818641 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d3716-7fd4-4dc0-bdec-01e8246ce8df admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.828542 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1404/2848] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:51.832910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.843069 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5b11-a5ff-4593-ac6c-5496ececf8f2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:51.847673 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:51.848322 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.848601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.848875 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] PATH_INFO: `/v3/projects/5f31299a7bd74105980480b171684977/users/50372d8baab24bfdba296b37b3fb4ce7/roles/bc2f8d451570479d9dd920a4c92b8ddf` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.850805 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f31299a7bd74105980480b171684977, user_id=50372d8baab24bfdba296b37b3fb4ce7, role_id=bc2f8d451570479d9dd920a4c92b8ddf)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:51.853492 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:51.864080 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1445/2849] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:51 2022] PUT /identity/v3/projects/5f31299a7bd74105980480b171684977/users/50372d8baab24bfdba296b37b3fb4ce7/roles/bc2f8d451570479d9dd920a4c92b8ddf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:51.873844 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:51.874129 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:51.874400 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:51.902900 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] MFA Rules not processed for user `50372d8baab24bfdba296b37b3fb4ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:51.904085 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:51.969948 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1405/2850] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:51.980680 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:51.982551 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:52.040591 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff2c07-6c41-4bf4-8334-8efa2a4d26a8 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:52.044292 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:52.044892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:52.045169 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:52.045438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:52.100590 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:52.102034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa2a8d-e0e1-4fb2-ba6d-27feace7b749 service cinder] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:52.111947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1446/2851] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:54.632795 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:54.693836 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5bdf2-d12d-4a5e-8bfa-30c6ebd73a20 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:54.698647 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:54.699280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:54.699594 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:54.699869 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] PATH_INFO: `/v3/users/afaa5dbe9e98497392b7999515e9528e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:54.700985 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=afaa5dbe9e98497392b7999515e9528e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:54.702911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:54.748552 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afaa5dbe9e98497392b7999515e9528e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:54.749194 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afaa5dbe9e98497392b7999515e9528e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:54.756369 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afaa5dbe9e98497392b7999515e9528e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:54.763444 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:54.771278 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afaa5dbe9e98497392b7999515e9528e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:54.778513 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afaa5dbe9e98497392b7999515e9528e'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:54.785094 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1406/2852] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:54 2022] DELETE /identity/v3/users/afaa5dbe9e98497392b7999515e9528e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:54.789689 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:54.790145 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:54.790262 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:54.808652 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:54.809857 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:54.868370 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1447/2853] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:54.878025 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:54.879459 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:54.936911 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f202-49d6-4651-8b6f-2c6839184dcc admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:54.941258 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:54.941883 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:54.942161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:54.942432 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:54.951729 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:54.953149 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:54.962532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1407/2854] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:00:55.232250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:55.233948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:55.248516 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728bdd8-3d77-43c9-af17-9ae1f8cb3c27 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:55.253663 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:55.254508 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:55.255047 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:55.255199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] PATH_INFO: `/v3/projects/fb1293005af84968832027558519c91b` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:55.256466 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1293005af84968832027558519c91b)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:55.258301 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:55.291619 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:INSERT=2,keystone:SELECT=96,keystone:DELETE=2 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:55.305901 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1293005af84968832027558519c91b', 'request_id': 'req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.311280 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1293005af84968832027558519c91b', 'request_id': 'req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.318150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1293005af84968832027558519c91b', 'request_id': 'req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.329335 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] The token cache is being invalidate because project fb1293005af84968832027558519c91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:55.329626 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.330257 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.331838 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1448/2855] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:55 2022] DELETE /identity/v3/projects/fb1293005af84968832027558519c91b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:55.335919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:55.345562 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cfe9-a149-44a2-a8dd-8b7049803676 service neutron] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:55.403251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:55.404009 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:55.404457 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:55.404915 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] PATH_INFO: `/v3/users/50372d8baab24bfdba296b37b3fb4ce7` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:55.406151 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=50372d8baab24bfdba296b37b3fb4ce7)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:55.408204 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:55.454610 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50372d8baab24bfdba296b37b3fb4ce7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.455254 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50372d8baab24bfdba296b37b3fb4ce7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.462468 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50372d8baab24bfdba296b37b3fb4ce7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.475071 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50372d8baab24bfdba296b37b3fb4ce7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.481939 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-904dc9d7-38d6-4bcb-b96f-252182662bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50372d8baab24bfdba296b37b3fb4ce7'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.488483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1408/2856] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:00:55 2022] DELETE /identity/v3/users/50372d8baab24bfdba296b37b3fb4ce7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:55.773945 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:55.775423 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:55.792435 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bdda7-3c39-4141-b7e0-bae4c01ccfa4 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:55.849418 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:55.850032 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:55.850254 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:55.850527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] PATH_INFO: `/v3/projects/5f31299a7bd74105980480b171684977` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:55.851678 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f31299a7bd74105980480b171684977)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:55.853628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:55.898520 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f31299a7bd74105980480b171684977', 'request_id': 'req-167f0152-203d-4e31-bd36-f8270c1f050c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.904021 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f31299a7bd74105980480b171684977', 'request_id': 'req-167f0152-203d-4e31-bd36-f8270c1f050c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.911720 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f31299a7bd74105980480b171684977', 'request_id': 'req-167f0152-203d-4e31-bd36-f8270c1f050c'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.922991 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] The token cache is being invalidate because project 5f31299a7bd74105980480b171684977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:00:55.923283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.923932 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-167f0152-203d-4e31-bd36-f8270c1f050c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:00:55.925978 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1449/2857] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:00:55 2022] DELETE /identity/v3/projects/5f31299a7bd74105980480b171684977 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:55.930887 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786f96-9241-4130-977a-46a861c623c9 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:55.931357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786f96-9241-4130-977a-46a861c623c9 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:55.931500 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19786f96-9241-4130-977a-46a861c623c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:55.939324 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:55.953601 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-19786f96-9241-4130-977a-46a861c623c9 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:55.954917 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-19786f96-9241-4130-977a-46a861c623c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.015411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1409/2858] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:56.024307 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.024602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.024880 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.045186 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:56.046415 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.056150 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1450/2859] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:56.060392 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19786f96-9241-4130-977a-46a861c623c9 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.070298 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19786f96-9241-4130-977a-46a861c623c9 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.074813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.075402 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.075703 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.075971 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] PATH_INFO: `/v3/domains` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.076599 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:00:56.076896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.078473 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.089710 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1410/2860] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:00:56.094106 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.104083 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a081f-6c41-4c06-96c2-b1048d86b50e None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.109359 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.109954 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.110235 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.110510 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.111133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.113723 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.140044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1451/2861] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:00:56 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:56.144209 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.153993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d80531-c8f9-467c-a5d6-b27c84300bea admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.209313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.209905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.210178 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.210445 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] PATH_INFO: `/v3/users` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.211047 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.212996 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.234312 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1411/2862] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:00:56 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:00:56.239028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.249473 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea46ca-c2cf-42aa-a388-cff51f81c9e1 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.254114 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.254703 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.254982 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.255253 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.255915 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.257602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.267429 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1452/2863] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:56.271575 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.281571 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63670b51-41cb-4b5f-86b8-b05208d05d27 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.286905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.287553 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.287834 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.288105 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.288739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.290363 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266f2a7-06c3-4553-ae4e-bf7761b7e8d1 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.300774 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1412/2864] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:00:56.304885 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.314927 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3770d5-fd7b-480d-a2ea-a154fd659f46 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.319609 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.320650 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.320878 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.321050 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] PATH_INFO: `/v3/projects/024bf3b8c67247c8bb6bfa06f3d7e96e/users/4815a603d8c642578956a6fac2efb432/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.324206 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:56.337910 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=024bf3b8c67247c8bb6bfa06f3d7e96e, user_id=4815a603d8c642578956a6fac2efb432, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.340602 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.353782 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1453/2865] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:00:56 2022] PUT /identity/v3/projects/024bf3b8c67247c8bb6bfa06f3d7e96e/users/4815a603d8c642578956a6fac2efb432/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:00:56.363955 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.364251 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.364555 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.391632 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] MFA Rules not processed for user `4815a603d8c642578956a6fac2efb432`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:00:56.392871 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.456899 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:00:56.463327 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1413/2866] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:00:56.475778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:00:56.477505 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:00:56.554434 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017dbf-2476-4adf-bf37-a32edad6fa7c admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:00:56.558133 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ae693a209514a79b82bc3f9ba50dccb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:00:56.558732 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:00:56.559012 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:00:56.559283 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:00:56.567518 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:00:56.568966 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:00:56.578066 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1454/2867] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:00:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:06.463762 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:06.573993 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:14.265537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:14.267150 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:14.338934 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6fde-3585-4206-8b58-db7fe7022744 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:14.343993 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:14.344665 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:14.344875 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:14.345148 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:14.346866 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:14.359155 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:14.360766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c76307-501e-4d84-ae02-8e2ad11d6d44 service placement] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:14.372158 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1414/2868] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:22.210841 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:22.272525 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ed36-5c63-456a-8222-5db5dc776660 service glance] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:22.277059 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:22.277679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.277958 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:22.278229 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] PATH_INFO: `/v3/users/4815a603d8c642578956a6fac2efb432` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:22.279345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4815a603d8c642578956a6fac2efb432)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:22.281308 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:22.325952 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4815a603d8c642578956a6fac2efb432'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.326536 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4815a603d8c642578956a6fac2efb432'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.333840 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4815a603d8c642578956a6fac2efb432'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.341495 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4815a603d8c642578956a6fac2efb432'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.353406 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4815a603d8c642578956a6fac2efb432'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.359127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1455/2869] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:01:22 2022] DELETE /identity/v3/users/4815a603d8c642578956a6fac2efb432 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 18:01:22.363763 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.364273 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:22.364483 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:22.384161 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:22.385355 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:22.442766 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1415/2870] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 18:01:22.453527 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:22.454956 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:22.513822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819100ff-fc80-4743-8929-78d5941096d9 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:22.517524 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:22.518127 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.518408 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:22.518679 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:22.528597 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:22.530044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d0586-bace-479a-9b08-14c0c7120682 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:22.539896 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1456/2871] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:22.804625 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:22.806045 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:22.817767 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd0118-87fb-4b0f-8c3b-033a9db1359f None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:22.822951 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:22.823644 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.823919 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:22.824187 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] PATH_INFO: `/v3/projects/024bf3b8c67247c8bb6bfa06f3d7e96e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:22.825313 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024bf3b8c67247c8bb6bfa06f3d7e96e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:22.827249 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:22.875077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024bf3b8c67247c8bb6bfa06f3d7e96e', 'request_id': 'req-151ae8e0-fc58-4687-9419-8f59e5604b40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.880885 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024bf3b8c67247c8bb6bfa06f3d7e96e', 'request_id': 'req-151ae8e0-fc58-4687-9419-8f59e5604b40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.888337 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024bf3b8c67247c8bb6bfa06f3d7e96e', 'request_id': 'req-151ae8e0-fc58-4687-9419-8f59e5604b40'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.899739 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] The token cache is being invalidate because project 024bf3b8c67247c8bb6bfa06f3d7e96e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:01:22.900024 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.900645 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-151ae8e0-fc58-4687-9419-8f59e5604b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:22.902113 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1416/2872] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:01:22 2022] DELETE /identity/v3/projects/024bf3b8c67247c8bb6bfa06f3d7e96e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:22.906947 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.907231 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:22.907530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:22.926224 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:22.927416 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:22.944722 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:22.990624 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1457/2873] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:22.999489 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:22.999797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.000069 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.020131 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:23.021339 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:23.030280 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1417/2874] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:23.034250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.044332 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbe2cf-ecb8-450b-99c2-c2596883cabc None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.048996 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.049601 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.049882 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.050184 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.050822 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:01:23.051125 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.052762 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.064659 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1458/2875] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:01:23.069509 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.078928 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14047bb-668c-4f07-8587-76a83915d6fc None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.083611 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.084201 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.085087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.085371 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.085989 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.087903 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.092310 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:23.115789 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1418/2876] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:01:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:01:23.119888 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.129804 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359887cc-0815-4f8f-8a7b-6606ad934675 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.191831 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.192348 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.192628 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.192924 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.193552 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.196374 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.216653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1459/2877] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:01:23 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 18:01:23.220820 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.231213 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25dc7b-611a-491c-a183-2b880f40434e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.235863 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.236482 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.236745 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.237017 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.237656 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.239428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.249771 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1419/2878] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:01:23.253767 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.264340 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a3084-0ab6-4276-a5d8-d3e19845a142 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.269138 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.269739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.270019 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.270305 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.270948 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.272568 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073818b-7c4f-4387-895c-49196afe367e admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.283163 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1460/2879] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:01:23.287092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.296941 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca5a86-bcda-49fc-ab7a-e05163c63c5e admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.301583 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.302224 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.302504 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.302778 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] PATH_INFO: `/v3/projects/1813facb7c794baab9d50d36faaa0035/users/08663425946644569bb13901fb6e169a/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.317533 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1813facb7c794baab9d50d36faaa0035, user_id=08663425946644569bb13901fb6e169a, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.321117 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.333498 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1420/2880] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:01:23 2022] PUT /identity/v3/projects/1813facb7c794baab9d50d36faaa0035/users/08663425946644569bb13901fb6e169a/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:23.343712 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.344009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.344284 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.372034 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] MFA Rules not processed for user `08663425946644569bb13901fb6e169a`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:23.373250 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:23.437000 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1461/2881] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:23.448847 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.450535 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:23.507138 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b1d6-d6b2-4605-b871-48f88677dde7 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.510830 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.511428 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.511734 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.512003 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.520361 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.521799 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327d97f-1a59-422a-899e-141471926605 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.531177 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1421/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:23.555701 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.574526 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:23.618582 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1311c-f612-4c23-a4f8-9e2a79237691 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.623461 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.624116 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.624392 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.624661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] PATH_INFO: `/v3/users/08663425946644569bb13901fb6e169a` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.625778 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08663425946644569bb13901fb6e169a)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.627763 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.673749 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08663425946644569bb13901fb6e169a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:23.674404 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08663425946644569bb13901fb6e169a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:23.682056 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08663425946644569bb13901fb6e169a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:23.689033 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08663425946644569bb13901fb6e169a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:23.701474 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08663425946644569bb13901fb6e169a'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:23.707176 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1462/2883] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:01:23 2022] DELETE /identity/v3/users/08663425946644569bb13901fb6e169a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:23.711723 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.712012 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.712291 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.731661 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:23.732882 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:23.791175 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1422/2884] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 18:01:23.800923 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:23.802354 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:23.859371 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a93a6a-0923-4c14-9260-b127d7df959f admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:23.863043 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:23.863667 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:23.863944 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:23.864209 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:23.873636 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:23.875049 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472d8bc-12b0-46e6-8f12-fcd8f31588f8 service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:23.884818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1463/2885] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:24.152818 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.154409 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:24.169223 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad88de-c03b-4b77-a29f-413cb55e19f6 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.173853 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.174532 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.174812 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.175087 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] PATH_INFO: `/v3/projects/1813facb7c794baab9d50d36faaa0035` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.177088 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1813facb7c794baab9d50d36faaa0035)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.179582 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.224797 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:24.226077 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1813facb7c794baab9d50d36faaa0035', 'request_id': 'req-118d5963-008b-4f15-a11e-461ff0360b39'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:24.232965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1813facb7c794baab9d50d36faaa0035', 'request_id': 'req-118d5963-008b-4f15-a11e-461ff0360b39'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:24.240411 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1813facb7c794baab9d50d36faaa0035', 'request_id': 'req-118d5963-008b-4f15-a11e-461ff0360b39'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:24.251805 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] The token cache is being invalidate because project 1813facb7c794baab9d50d36faaa0035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:01:24.252092 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:24.252721 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-118d5963-008b-4f15-a11e-461ff0360b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:24.254200 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1423/2886] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:01:24 2022] DELETE /identity/v3/projects/1813facb7c794baab9d50d36faaa0035 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:24.259739 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e226723-f922-431d-844d-09729dc78c2a None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.260199 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e226723-f922-431d-844d-09729dc78c2a None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.260337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e226723-f922-431d-844d-09729dc78c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.280458 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-3e226723-f922-431d-844d-09729dc78c2a None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:24.281674 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-3e226723-f922-431d-844d-09729dc78c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:24.314698 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:24.341755 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1464/2887] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:24.349466 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.349748 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.350022 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.371624 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:24.372854 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:24.380643 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1424/2888] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:24.384892 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e226723-f922-431d-844d-09729dc78c2a None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.394578 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e226723-f922-431d-844d-09729dc78c2a None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.399160 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.399795 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.400075 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.400345 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] PATH_INFO: `/v3/domains` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.400988 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 18:01:24.401319 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.402919 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.414606 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1465/2889] 10.222.0.27 () {58 vars in 1153 bytes} [Wed Aug 24 18:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 18:01:24.418709 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.427826 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ce70d-1684-4e68-a1d8-1c0b398d012b None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.433227 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.433813 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.434089 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.434357 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] PATH_INFO: `/v3/projects` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.434965 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.436873 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.463174 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1425/2890] 10.222.0.27 () {60 vars in 1153 bytes} [Wed Aug 24 18:01:24 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 18:01:24.467171 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.477543 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfb07b-e2ad-4e81-8eed-03986e8b5772 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.533603 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.534214 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.534515 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.534788 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] PATH_INFO: `/v3/users` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.535421 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.537372 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.558528 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1466/2891] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Aug 24 18:01:24 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 18:01:24.562697 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.573196 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212886fc-dd9b-4caf-be2b-6cc99cde9473 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.577781 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.578368 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.578642 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.578905 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.579566 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.581137 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.592253 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1426/2892] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:01:24.596337 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.606921 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51a85a-754c-4e8b-bfd0-052238c398f2 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.611661 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.612251 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.612530 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.612798 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.613432 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.615031 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb432acd-2935-4087-9927-6a82faa83fbc admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.626009 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1467/2893] 10.222.0.27 () {58 vars in 1122 bytes} [Wed Aug 24 18:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 18:01:24.630039 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.640091 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958025f5-8a42-4c57-ab93-956ac9f60f7a admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.645166 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.645896 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] REQUEST_METHOD: `PUT` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.646193 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.646452 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] PATH_INFO: `/v3/projects/c698342473a448f09b70eb7e2fe35714/users/4f891819c25a4e20b600d9a2ad1378f7/roles/7e7dcf0e0b644abb9ae0549320fd777e` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.661518 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c698342473a448f09b70eb7e2fe35714, user_id=4f891819c25a4e20b600d9a2ad1378f7, role_id=7e7dcf0e0b644abb9ae0549320fd777e)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.664852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.677217 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1427/2894] 10.222.0.27 () {60 vars in 1483 bytes} [Wed Aug 24 18:01:24 2022] PUT /identity/v3/projects/c698342473a448f09b70eb7e2fe35714/users/4f891819c25a4e20b600d9a2ad1378f7/roles/7e7dcf0e0b644abb9ae0549320fd777e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:24.688379 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] REQUEST_METHOD: `POST` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.688670 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.688951 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.716889 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.auth.core [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] MFA Rules not processed for user `4f891819c25a4e20b600d9a2ad1378f7`. Rule list: `[]` (Enabled: `True`). {{(pid=87988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:24.718112 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:24.780835 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1468/2895] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:24.792043 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:24.793728 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:24.851359 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db447397-377e-4248-a7e9-b7dd33156994 admin admin] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:24.855006 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e690af7e92f48efb0cbc9dee2c87ba4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:24.855639 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] REQUEST_METHOD: `GET` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:24.855916 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:24.856185 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:24.864264 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:24.865689 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcb713-0d07-41dc-b560-4517a3c76292 service cinder] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:24.874726 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1428/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:34.778002 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:34.787028 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:34.848365 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787177db-e1e2-4caf-ab8a-83f109120930 None None] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:34.853537 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:34.854158 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:34.854438 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:34.854709 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] PATH_INFO: `/v3/users/4f891819c25a4e20b600d9a2ad1378f7` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:34.855849 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f891819c25a4e20b600d9a2ad1378f7)` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:34.857781 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:34.870507 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:34.902771 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f891819c25a4e20b600d9a2ad1378f7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:34.903410 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f891819c25a4e20b600d9a2ad1378f7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:34.910375 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f891819c25a4e20b600d9a2ad1378f7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:34.917983 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f891819c25a4e20b600d9a2ad1378f7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:34.929764 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.notifications [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f891819c25a4e20b600d9a2ad1378f7'} {{(pid=87988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:34.936055 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1469/2897] 10.222.0.27 () {58 vars in 1224 bytes} [Wed Aug 24 18:01:34 2022] DELETE /identity/v3/users/4f891819c25a4e20b600d9a2ad1378f7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:34.940537 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] REQUEST_METHOD: `POST` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:34.940823 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:34.941101 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:34.960673 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.auth.core [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] MFA Rules not processed for user `6c7d397143574599b62017bc352e1183`. Rule list: `[]` (Enabled: `True`). {{(pid=87987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 18:01:34.962461 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:35.018421 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1429/2898] 10.222.0.27 () {58 vars in 958 bytes} [Wed Aug 24 18:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 18:01:35.028704 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:35.030104 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:35.086173 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111edd-5a53-4dce-bfb6-f666439ef474 admin admin] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:35.090391 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c18cb9acd43548ed9c8efe69f77b8df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:35.090989 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:35.091269 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:35.091570 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:35.100420 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:35.101818 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:35.112044 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1470/2899] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:01:35.383037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] Authenticating user token {{(pid=87987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:01:35.385246 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.fernet_utils [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:01:35.397252 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bdbda-9b77-4f2e-a9ff-69cdad02c7c0 None None] Validating token access rules against request {{(pid=87987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:01:35.401785 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c7d397143574599b62017bc352e1183', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0a52b9c017649d7b6608817068a47f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:01:35.402424 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:01:35.402699 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:01:35.402969 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] PATH_INFO: `/v3/projects/c698342473a448f09b70eb7e2fe35714` {{(pid=87987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:01:35.404096 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c698342473a448f09b70eb7e2fe35714)` {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:01:35.406037 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] RBAC: Authorization granted {{(pid=87987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:01:35.453696 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c698342473a448f09b70eb7e2fe35714', 'request_id': 'req-aa4cbd50-1b02-4425-af0c-9014396cee1d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:35.459286 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c698342473a448f09b70eb7e2fe35714', 'request_id': 'req-aa4cbd50-1b02-4425-af0c-9014396cee1d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:35.467966 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c698342473a448f09b70eb7e2fe35714', 'request_id': 'req-aa4cbd50-1b02-4425-af0c-9014396cee1d'} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:35.479852 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] The token cache is being invalidate because project c698342473a448f09b70eb7e2fe35714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 18:01:35.480143 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:35.480776 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG keystone.notifications [None req-aa4cbd50-1b02-4425-af0c-9014396cee1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 18:01:35.482261 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: [pid: 87987|app: 0|req: 1430/2900] 10.222.0.27 () {58 vars in 1233 bytes} [Wed Aug 24 18:01:35 2022] DELETE /identity/v3/projects/c698342473a448f09b70eb7e2fe35714 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 18:01:45.107454 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:01:45.476329 dv-cinder-840525-3-iscsi devstack@keystone.service[87987]: DEBUG dbcounter [-] [87987] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 18:06:20.269990 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] Authenticating user token {{(pid=87988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 18:06:20.271457 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:06:20.408038 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f8a57-4879-4faa-8382-34ece8ab73ec service neutron] Validating token access rules against request {{(pid=87988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 18:06:20.466477 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb029fc999c644408f292f85e4ab114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff81e911885e43ed86627df1e1a85893', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 18:06:20.467081 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] REQUEST_METHOD: `GET` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 18:06:20.467356 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] SCRIPT_NAME: `/identity` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 18:06:20.467653 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 18:06:20.469377 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.fernet_utils [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 18:06:20.481082 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 18:06:20.482743 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06728c-dd13-4028-8e9a-2f160fb7e967 service placement] RBAC: Authorization granted {{(pid=87988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 18:06:20.493959 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: [pid: 87988|app: 0|req: 1471/2901] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Aug 24 18:06:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 18:06:30.488874 dv-cinder-840525-3-iscsi devstack@keystone.service[87988]: DEBUG dbcounter [-] [87988] Writing DB stats keystone:SELECT=89 {{(pid=87988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}